[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from RaftConsensusStressITest
[ RUN ] RaftConsensusStressITest.RemoveReplaceInCycle
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20240503 08:12:52.518555 2103 test_util.cc:268] Using random seed: 1229263469
I20240503 08:12:52.529855 2103 ts_itest-base.cc:112] Starting cluster with:
I20240503 08:12:52.529942 2103 ts_itest-base.cc:113] --------------
I20240503 08:12:52.529994 2103 ts_itest-base.cc:114] 6 tablet servers
I20240503 08:12:52.530053 2103 ts_itest-base.cc:115] 3 replicas per TS
I20240503 08:12:52.530103 2103 ts_itest-base.cc:116] --------------
2024-05-03T08:12:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-03T08:12:52Z Disabled control of system clock
I20240503 08:12:52.552481 2103 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV_7T9/build/asan/bin/kudu
/tmp/dist-test-tasksV_7T9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.13.254:46769
--webserver_interface=127.2.13.254
--webserver_port=0
--builtin_ntp_servers=127.2.13.212:33833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.13.254:46769
--minloglevel=2
--raft_prepare_replacement_before_eviction=true
--max_create_tablets_per_ts=20 with env {}
I20240503 08:12:52.946718 2103 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV_7T9/build/asan/bin/kudu as pid 2115
I20240503 08:12:52.946923 2103 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240503 08:12:53.041831 2103 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV_7T9/build/asan/bin/kudu
/tmp/dist-test-tasksV_7T9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.13.193:0
--local_ip_for_outbound_sockets=127.2.13.193
--webserver_interface=127.2.13.193
--webserver_port=0
--tserver_master_addrs=127.2.13.254:46769
--builtin_ntp_servers=127.2.13.212:33833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--minloglevel=2
--tablet_copy_early_session_timeout_prob=0.125
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=2
--consensus_rpc_timeout_ms=2000
--heartbeat_interval_ms=500
--raft_heartbeat_interval_ms=200 with env {}
I20240503 08:12:53.478971 2103 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV_7T9/build/asan/bin/kudu as pid 2205
I20240503 08:12:53.479230 2103 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240503 08:12:53.482688 2103 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV_7T9/build/asan/bin/kudu
/tmp/dist-test-tasksV_7T9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.13.194:0
--local_ip_for_outbound_sockets=127.2.13.194
--webserver_interface=127.2.13.194
--webserver_port=0
--tserver_master_addrs=127.2.13.254:46769
--builtin_ntp_servers=127.2.13.212:33833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--minloglevel=2
--tablet_copy_early_session_timeout_prob=0.125
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=2
--consensus_rpc_timeout_ms=2000
--heartbeat_interval_ms=500
--raft_heartbeat_interval_ms=200 with env {}
I20240503 08:12:53.907127 2103 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV_7T9/build/asan/bin/kudu as pid 2336
I20240503 08:12:53.907321 2103 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240503 08:12:53.910684 2103 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV_7T9/build/asan/bin/kudu
/tmp/dist-test-tasksV_7T9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.13.195:0
--local_ip_for_outbound_sockets=127.2.13.195
--webserver_interface=127.2.13.195
--webserver_port=0
--tserver_master_addrs=127.2.13.254:46769
--builtin_ntp_servers=127.2.13.212:33833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--minloglevel=2
--tablet_copy_early_session_timeout_prob=0.125
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=2
--consensus_rpc_timeout_ms=2000
--heartbeat_interval_ms=500
--raft_heartbeat_interval_ms=200 with env {}
I20240503 08:12:54.346920 2103 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV_7T9/build/asan/bin/kudu as pid 2466
I20240503 08:12:54.347127 2103 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240503 08:12:54.350311 2103 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV_7T9/build/asan/bin/kudu
/tmp/dist-test-tasksV_7T9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.13.196:0
--local_ip_for_outbound_sockets=127.2.13.196
--webserver_interface=127.2.13.196
--webserver_port=0
--tserver_master_addrs=127.2.13.254:46769
--builtin_ntp_servers=127.2.13.212:33833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--minloglevel=2
--tablet_copy_early_session_timeout_prob=0.125
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=2
--consensus_rpc_timeout_ms=2000
--heartbeat_interval_ms=500
--raft_heartbeat_interval_ms=200 with env {}
I20240503 08:12:54.775048 2103 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV_7T9/build/asan/bin/kudu as pid 2596
I20240503 08:12:54.775254 2103 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240503 08:12:54.778566 2103 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV_7T9/build/asan/bin/kudu
/tmp/dist-test-tasksV_7T9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.13.197:0
--local_ip_for_outbound_sockets=127.2.13.197
--webserver_interface=127.2.13.197
--webserver_port=0
--tserver_master_addrs=127.2.13.254:46769
--builtin_ntp_servers=127.2.13.212:33833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--minloglevel=2
--tablet_copy_early_session_timeout_prob=0.125
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=2
--consensus_rpc_timeout_ms=2000
--heartbeat_interval_ms=500
--raft_heartbeat_interval_ms=200 with env {}
I20240503 08:12:55.193413 2103 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV_7T9/build/asan/bin/kudu as pid 2726
I20240503 08:12:55.193620 2103 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240503 08:12:55.197095 2103 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV_7T9/build/asan/bin/kudu
/tmp/dist-test-tasksV_7T9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.13.198:0
--local_ip_for_outbound_sockets=127.2.13.198
--webserver_interface=127.2.13.198
--webserver_port=0
--tserver_master_addrs=127.2.13.254:46769
--builtin_ntp_servers=127.2.13.212:33833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--minloglevel=2
--tablet_copy_early_session_timeout_prob=0.125
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=2
--consensus_rpc_timeout_ms=2000
--heartbeat_interval_ms=500
--raft_heartbeat_interval_ms=200 with env {}
I20240503 08:12:55.641633 2103 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV_7T9/build/asan/bin/kudu as pid 2856
I20240503 08:12:55.641775 2103 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/wal/instance
I20240503 08:12:55.658030 2103 external_mini_cluster.cc:930] 6 TS(s) registered with all masters
I20240503 08:12:55.903786 2103 external_mini_cluster.cc:930] 6 TS(s) registered with all masters
I20240503 08:12:55.906447 2103 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 4d907d33630a4dfc8b548e94bc03f7fa to finish bootstrapping
I20240503 08:12:55.915089 2103 ts_itest-base.cc:233] Waiting for 1 tablets on tserver c120f4d1df6c4e16b63556d1f64f8a99 to finish bootstrapping
I20240503 08:12:55.922030 2103 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 646555016bb14084999bff39175a18f0 to finish bootstrapping
I20240503 08:12:55.929733 2103 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 40e91970c83a45debf7f4c6a20ee6d93 to finish bootstrapping
I20240503 08:12:55.936254 2103 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 3d72f5d7124c4af5a05043929be3f350 to finish bootstrapping
I20240503 08:12:55.942996 2103 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 9122a9c942514b92aa63586e9538fc25 to finish bootstrapping
I20240503 08:12:55.952134 2103 test_util.cc:268] Using random seed: 1232697055
W20240503 08:13:16.319682 3751 negotiation.cc:337] Failed RPC negotiation. Trace:
0503 08:13:13.307737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.13.198:37463 (local address 127.0.0.1:41880)
0503 08:13:13.308018 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0503 08:13:13.308089 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0503 08:13:13.308206 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0503 08:13:16.317916 (+3009710us) negotiation.cc:327] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.2.13.198:37463: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":130}
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 2 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 3 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 8 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.198:37463: Timed out: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.198:37463: received 0 of 4 requested bytes (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_server_key_size | 768 | experimental | all 6 server(s) checked
local_ip_for_outbound_sockets | 127.2.13.193 | experimental | 127.2.13.193:33069
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | all 6 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 6 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 6 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 6 server(s) checked
server_dump_info_format | pb | hidden | all 6 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.2.13.193:33069
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | all 6 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 6 server(s) checked
time_source | builtin | all 6 server(s) checked
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.196:41833, tserver@127.2.13.197:46671, and 3 other server(s)
Tablet Summary
Tablet 70d88dbe835c4ba99a8f22477926a15f of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
646555016bb14084999bff39175a18f0 (127.2.13.195:46293): RUNNING
9122a9c942514b92aa63586e9538fc25 (127.2.13.198:37463): TS unavailable [LEADER]
c120f4d1df6c4e16b63556d1f64f8a99 (127.2.13.194:37017): RUNNING
All reported replicas are:
A = 646555016bb14084999bff39175a18f0
B = 9122a9c942514b92aa63586e9538fc25
C = c120f4d1df6c4e16b63556d1f64f8a99
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B* C | 1 | 44 | Yes
B | [config not available] | | |
C | A B* C | 1 | 44 | Yes
Tablet fd5225972315419d89430cd3c5da941c of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
c120f4d1df6c4e16b63556d1f64f8a99 (127.2.13.194:37017): RUNNING [LEADER]
646555016bb14084999bff39175a18f0 (127.2.13.195:46293): RUNNING
9122a9c942514b92aa63586e9538fc25 (127.2.13.198:37463): TS unavailable
All reported replicas are:
A = c120f4d1df6c4e16b63556d1f64f8a99
B = 646555016bb14084999bff39175a18f0
C = 9122a9c942514b92aa63586e9538fc25
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 2 | 44 | Yes
B | A* B C | 2 | 44 | Yes
C | [config not available] | | |
Tablet 1e4f8cf1ceff4c1b805cb05eb2c57918 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
3d72f5d7124c4af5a05043929be3f350 (127.2.13.197:46671): RUNNING
9122a9c942514b92aa63586e9538fc25 (127.2.13.198:37463): TS unavailable [LEADER]
4d907d33630a4dfc8b548e94bc03f7fa (127.2.13.193:33069): RUNNING
All reported replicas are:
A = 3d72f5d7124c4af5a05043929be3f350
B = 9122a9c942514b92aa63586e9538fc25
C = 4d907d33630a4dfc8b548e94bc03f7fa
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B* C | 2 | 45 | Yes
B | [config not available] | | |
C | A B* C | 2 | 45 | Yes
Tablet 523bf734b7664181a96a1ff4b5767604 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
c120f4d1df6c4e16b63556d1f64f8a99 (127.2.13.194:37017): RUNNING [LEADER]
646555016bb14084999bff39175a18f0 (127.2.13.195:46293): RUNNING
9122a9c942514b92aa63586e9538fc25 (127.2.13.198:37463): TS unavailable
All reported replicas are:
A = c120f4d1df6c4e16b63556d1f64f8a99
B = 646555016bb14084999bff39175a18f0
C = 9122a9c942514b92aa63586e9538fc25
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 2 | 41 | Yes
B | A* B C | 2 | 41 | Yes
C | [config not available] | | |
Tablet aa9c0fdabcfc4d10b26fd1c05043ac44 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
3d72f5d7124c4af5a05043929be3f350 (127.2.13.197:46671): RUNNING [LEADER]
9122a9c942514b92aa63586e9538fc25 (127.2.13.198:37463): TS unavailable
4d907d33630a4dfc8b548e94bc03f7fa (127.2.13.193:33069): RUNNING
All reported replicas are:
A = 3d72f5d7124c4af5a05043929be3f350
B = 9122a9c942514b92aa63586e9538fc25
C = 4d907d33630a4dfc8b548e94bc03f7fa
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 1 | 43 | Yes
B | [config not available] | | |
C | A* B C | 1 | 43 | Yes
Tablet b7b21c8ecf024c01a92c353394024285 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
c120f4d1df6c4e16b63556d1f64f8a99 (127.2.13.194:37017): RUNNING
646555016bb14084999bff39175a18f0 (127.2.13.195:46293): RUNNING [LEADER]
9122a9c942514b92aa63586e9538fc25 (127.2.13.198:37463): TS unavailable
All reported replicas are:
A = c120f4d1df6c4e16b63556d1f64f8a99
B = 646555016bb14084999bff39175a18f0
C = 9122a9c942514b92aa63586e9538fc25
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B* C | 1 | 44 | Yes
B | A B* C | 1 | 44 | Yes
C | [config not available] | | |
Tablet 4ccbb1e52c7d47aa8db0fe2165f98ecb of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
3d72f5d7124c4af5a05043929be3f350 (127.2.13.197:46671): RUNNING
9122a9c942514b92aa63586e9538fc25 (127.2.13.198:37463): TS unavailable [LEADER]
4d907d33630a4dfc8b548e94bc03f7fa (127.2.13.193:33069): RUNNING
All reported replicas are:
A = 3d72f5d7124c4af5a05043929be3f350
B = 9122a9c942514b92aa63586e9538fc25
C = 4d907d33630a4dfc8b548e94bc03f7fa
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B* C | 1 | 44 | Yes
B | [config not available] | | |
C | A B* C | 1 | 44 | Yes
Tablet e40655007d784cdb810c61e186e98abe of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
c120f4d1df6c4e16b63556d1f64f8a99 (127.2.13.194:37017): RUNNING
646555016bb14084999bff39175a18f0 (127.2.13.195:46293): RUNNING [LEADER]
9122a9c942514b92aa63586e9538fc25 (127.2.13.198:37463): TS unavailable
All reported replicas are:
A = c120f4d1df6c4e16b63556d1f64f8a99
B = 646555016bb14084999bff39175a18f0
C = 9122a9c942514b92aa63586e9538fc25
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B* C | 2 | 41 | Yes
B | A B* C | 2 | 41 | Yes
C | [config not available] | | |
Tablet eeb1cdb4349f4b62add4a02741ae7d58 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
3d72f5d7124c4af5a05043929be3f350 (127.2.13.197:46671): RUNNING
9122a9c942514b92aa63586e9538fc25 (127.2.13.198:37463): TS unavailable [LEADER]
4d907d33630a4dfc8b548e94bc03f7fa (127.2.13.193:33069): RUNNING
All reported replicas are:
A = 3d72f5d7124c4af5a05043929be3f350
B = 9122a9c942514b92aa63586e9538fc25
C = 4d907d33630a4dfc8b548e94bc03f7fa
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B* C | 2 | 44 | Yes
B | [config not available] | | |
C | A B* C | 2 | 44 | Yes
Tablet 7f04446f1405417ea781f0ddc8b27b68 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
c120f4d1df6c4e16b63556d1f64f8a99 (127.2.13.194:37017): RUNNING [LEADER]
646555016bb14084999bff39175a18f0 (127.2.13.195:46293): RUNNING
9122a9c942514b92aa63586e9538fc25 (127.2.13.198:37463): TS unavailable
All reported replicas are:
A = c120f4d1df6c4e16b63556d1f64f8a99
B = 646555016bb14084999bff39175a18f0
C = 9122a9c942514b92aa63586e9538fc25
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 1 | 45 | Yes
B | A* B C | 1 | 45 | Yes
C | [config not available] | | |
Tablet a7c4443c00da46f084c2d1fd81aaf261 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
3d72f5d7124c4af5a05043929be3f350 (127.2.13.197:46671): RUNNING [LEADER]
9122a9c942514b92aa63586e9538fc25 (127.2.13.198:37463): TS unavailable
4d907d33630a4dfc8b548e94bc03f7fa (127.2.13.193:33069): RUNNING
All reported replicas are:
A = 3d72f5d7124c4af5a05043929be3f350
B = 9122a9c942514b92aa63586e9538fc25
C = 4d907d33630a4dfc8b548e94bc03f7fa
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 2 | 45 | Yes
B | [config not available] | | |
C | A* B C | 2 | 45 | Yes
Tablet 0570e726d9bb426581689d5cb6d88177 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
c120f4d1df6c4e16b63556d1f64f8a99 (127.2.13.194:37017): RUNNING
646555016bb14084999bff39175a18f0 (127.2.13.195:46293): RUNNING [LEADER]
9122a9c942514b92aa63586e9538fc25 (127.2.13.198:37463): TS unavailable
All reported replicas are:
A = c120f4d1df6c4e16b63556d1f64f8a99
B = 646555016bb14084999bff39175a18f0
C = 9122a9c942514b92aa63586e9538fc25
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B* C | 2 | 46 | Yes
B | A B* C | 2 | 46 | Yes
C | [config not available] | | |
Tablet 59cded2638f24227bae24907a2c1c586 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
3d72f5d7124c4af5a05043929be3f350 (127.2.13.197:46671): RUNNING
9122a9c942514b92aa63586e9538fc25 (127.2.13.198:37463): TS unavailable [LEADER]
4d907d33630a4dfc8b548e94bc03f7fa (127.2.13.193:33069): RUNNING
All reported replicas are:
A = 3d72f5d7124c4af5a05043929be3f350
B = 9122a9c942514b92aa63586e9538fc25
C = 4d907d33630a4dfc8b548e94bc03f7fa
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B* C | 1 | 43 | Yes
B | [config not available] | | |
C | A B* C | 1 | 43 | Yes
Tablet ce371c0c13444122b13ddd928e3f502f of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
c120f4d1df6c4e16b63556d1f64f8a99 (127.2.13.194:37017): RUNNING [LEADER]
646555016bb14084999bff39175a18f0 (127.2.13.195:46293): RUNNING
9122a9c942514b92aa63586e9538fc25 (127.2.13.198:37463): TS unavailable
All reported replicas are:
A = c120f4d1df6c4e16b63556d1f64f8a99
B = 646555016bb14084999bff39175a18f0
C = 9122a9c942514b92aa63586e9538fc25
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 3 | 41 | Yes
B | A* B C | 3 | 41 | Yes
C | [config not available] | | |
Tablet fc505438fcd54ddc8f32dd22a13390a9 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
3d72f5d7124c4af5a05043929be3f350 (127.2.13.197:46671): RUNNING
9122a9c942514b92aa63586e9538fc25 (127.2.13.198:37463): TS unavailable [LEADER]
4d907d33630a4dfc8b548e94bc03f7fa (127.2.13.193:33069): RUNNING
All reported replicas are:
A = 3d72f5d7124c4af5a05043929be3f350
B = 9122a9c942514b92aa63586e9538fc25
C = 4d907d33630a4dfc8b548e94bc03f7fa
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B* C | 1 | 44 | Yes
B | [config not available] | | |
C | A B* C | 1 | 44 | Yes
Tablet a82d80a03df24e0a825c422362effe14 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
c120f4d1df6c4e16b63556d1f64f8a99 (127.2.13.194:37017): RUNNING [LEADER]
646555016bb14084999bff39175a18f0 (127.2.13.195:46293): RUNNING
9122a9c942514b92aa63586e9538fc25 (127.2.13.198:37463): TS unavailable
All reported replicas are:
A = c120f4d1df6c4e16b63556d1f64f8a99
B = 646555016bb14084999bff39175a18f0
C = 9122a9c942514b92aa63586e9538fc25
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 2 | 44 | Yes
B | A* B C | 2 | 44 | Yes
C | [config not available] | | |
Tablet a273db7b6ae9458ca8e44c665bfb2504 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
3d72f5d7124c4af5a05043929be3f350 (127.2.13.197:46671): RUNNING
9122a9c942514b92aa63586e9538fc25 (127.2.13.198:37463): TS unavailable [LEADER]
4d907d33630a4dfc8b548e94bc03f7fa (127.2.13.193:33069): RUNNING
All reported replicas are:
A = 3d72f5d7124c4af5a05043929be3f350
B = 9122a9c942514b92aa63586e9538fc25
C = 4d907d33630a4dfc8b548e94bc03f7fa
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B* C | 1 | 45 | Yes
B | [config not available] | | |
C | A B* C | 1 | 45 | Yes
Tablet 8c03af9f7884408d807c92c6cd6c0888 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
c120f4d1df6c4e16b63556d1f64f8a99 (127.2.13.194:37017): RUNNING [LEADER]
646555016bb14084999bff39175a18f0 (127.2.13.195:46293): RUNNING
9122a9c942514b92aa63586e9538fc25 (127.2.13.198:37463): TS unavailable
All reported replicas are:
A = c120f4d1df6c4e16b63556d1f64f8a99
B = 646555016bb14084999bff39175a18f0
C = 9122a9c942514b92aa63586e9538fc25
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 4 | 45 | Yes
B | A* B C | 4 | 45 | Yes
C | [config not available] | | |
Tablet c4be050ef4ec4fa8a3eb04c3fad2993b of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
3d72f5d7124c4af5a05043929be3f350 (127.2.13.197:46671): RUNNING
9122a9c942514b92aa63586e9538fc25 (127.2.13.198:37463): TS unavailable [LEADER]
4d907d33630a4dfc8b548e94bc03f7fa (127.2.13.193:33069): RUNNING
All reported replicas are:
A = 3d72f5d7124c4af5a05043929be3f350
B = 9122a9c942514b92aa63586e9538fc25
C = 4d907d33630a4dfc8b548e94bc03f7fa
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B* C | 2 | 43 | Yes
B | [config not available] | | |
C | A B* C | 2 | 43 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+------------------+---------------+---------+------------+------------------+-------------
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
RemoveReplaceInCycle | 3 | UNDER_REPLICATED | 20 | 1 | 0 | 19 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 10
Median | 11
Third Quartile | 12
Maximum | 19
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | 19
Small | 40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
Corruption: table consistency check error: 1 out of 2 table(s) are not healthy
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 4 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | HEALTHY | <none> | 5 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 3 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 0 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 9 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_server_key_size | 768 | experimental | all 6 server(s) checked
local_ip_for_outbound_sockets | 127.2.13.193 | experimental | 127.2.13.193:33069
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | all 6 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 6 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 6 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 6 server(s) checked
server_dump_info_format | pb | hidden | all 6 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.2.13.193:33069
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | all 6 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 6 server(s) checked
time_source | builtin | all 6 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 7 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 10
Median | 11
Third Quartile | 12
Maximum | 20
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | 20
Small | 40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20240503 08:13:27.237370 2103 raft_consensus_stress-itest.cc:258] completed iteration 0
W20240503 08:13:42.686161 5055 negotiation.cc:337] Failed RPC negotiation. Trace:
0503 08:13:39.526339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.13.196:41833 (local address 127.0.0.1:43256)
0503 08:13:39.558200 (+ 31861us) negotiation.cc:107] Waiting for socket to connect
0503 08:13:39.558220 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0503 08:13:39.558337 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0503 08:13:42.685811 (+3127474us) negotiation.cc:327] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.2.13.196:41833: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":31742,"thread_start_us":39226,"threads_started":1}
F20240503 08:13:43.634166 5130 trace_metrics.cc:68] Check failed: g_intern_map->size() < 100 (100 vs. 100) Too many interned strings: (MaintenanceMgr.queue_time_us, MaintenanceMgr.queue_time_us) (MaintenanceMgr.run_cpu_time_us, MaintenanceMgr.run_cpu_time_us) (MaintenanceMgr.run_wall_time_us, MaintenanceMgr.run_wall_time_us) (apply.queue_time_us, apply.queue_time_us) (apply.run_cpu_time_us, apply.run_cpu_time_us) (apply.run_wall_time_us, apply.run_wall_time_us) (client-negotiator.queue_time_us, client-negotiator.queue_time_us) (client-negotiator.run_cpu_time_us, client-negotiator.run_cpu_time_us) (client-negotiator.run_wall_time_us, client-negotiator.run_wall_time_us) (compiler_manager_pool.queue_time_us, compiler_manager_pool.queue_time_us) (compiler_manager_pool.run_cpu_time_us, compiler_manager_pool.run_cpu_time_us) (compiler_manager_pool.run_wall_time_us, compiler_manager_pool.run_wall_time_us) (dirs.queue_time_us, dirs.queue_time_us) (dirs.run_cpu_time_us, dirs.run_cpu_time_us) (dirs.run_wall_time_us, dirs.run_wall_time_us) (dns-resolver.queue_time_us, dns-resolver.queue_time_us) (dns-resolver.run_cpu_time_us, dns-resolver.run_cpu_time_us) (dns-resolver.run_wall_time_us, dns-resolver.run_wall_time_us) (log-alloc.queue_time_us, log-alloc.queue_time_us) (log-alloc.run_cpu_time_us, log-alloc.run_cpu_time_us) (log-alloc.run_wall_time_us, log-alloc.run_wall_time_us) (prepare.queue_time_us, prepare.queue_time_us) (prepare.run_cpu_time_us, prepare.run_cpu_time_us) (prepare.run_wall_time_us, prepare.run_wall_time_us) (raft.queue_time_us, raft.queue_time_us) (raft.run_cpu_time_us, raft.run_cpu_time_us) (raft.run_wall_time_us, raft.run_wall_time_us) (repair_pool.queue_time_us, repair_pool.queue_time_us) (repair_pool.run_cpu_time_us, repair_pool.run_cpu_time_us) (repair_pool.run_wall_time_us, repair_pool.run_wall_time_us) (server-negotiator.queue_time_us, server-negotiator.queue_time_us) (server-negotiator.run_cpu_time_us, server-negotiator.run_cpu_time_us) (server-negotiator.run_wall_time_us, server-negotiator.run_wall_time_us) (tablet-copy.queue_time_us, tablet-copy.queue_time_us) (tablet-copy.run_cpu_time_us, tablet-copy.run_cpu_time_us) (tablet-copy.run_wall_time_us, tablet-copy.run_wall_time_us) (tablet-delete.queue_time_us, tablet-delete.queue_time_us) (tablet-delete.run_cpu_time_us, tablet-delete.run_cpu_time_us) (tablet-delete.run_wall_time_us, tablet-delete.run_wall_time_us) (tablet-download-pool-0570e726d9bb426581689d5cb6d88177.queue_time_us, tablet-download-pool-0570e726d9bb426581689d5cb6d88177.queue_time_us) (tablet-download-pool-0570e726d9bb426581689d5cb6d88177.run_cpu_time_us, tablet-download-pool-0570e726d9bb426581689d5cb6d88177.run_cpu_time_us) (tablet-download-pool-0570e726d9bb426581689d5cb6d88177.run_wall_time_us, tablet-download-pool-0570e726d9bb426581689d5cb6d88177.run_wall_time_us) (tablet-download-pool-0f5de6a6722740af83fa8194084edbc0.queue_time_us, tablet-download-pool-0f5de6a6722740af83fa8194084edbc0.queue_time_us) (tablet-download-pool-0f5de6a6722740af83fa8194084edbc0.run_cpu_time_us, tablet-download-pool-0f5de6a6722740af83fa8194084edbc0.run_cpu_time_us) (tablet-download-pool-0f5de6a6722740af83fa8194084edbc0.run_wall_time_us, tablet-download-pool-0f5de6a6722740af83fa8194084edbc0.run_wall_time_us) (tablet-download-pool-1e4f8cf1ceff4c1b805cb05eb2c57918.queue_time_us, tablet-download-pool-1e4f8cf1ceff4c1b805cb05eb2c57918.queue_time_us) (tablet-download-pool-1e4f8cf1ceff4c1b805cb05eb2c57918.run_cpu_time_us, tablet-download-pool-1e4f8cf1ceff4c1b805cb05eb2c57918.run_cpu_time_us) (tablet-download-pool-1e4f8cf1ceff4c1b805cb05eb2c57918.run_wall_time_us, tablet-download-pool-1e4f8cf1ceff4c1b805cb05eb2c57918.run_wall_time_us) (tablet-download-pool-26ba4bcaae574ccbaa5c80e81e79b37a.queue_time_us, tablet-download-pool-26ba4bcaae574ccbaa5c80e81e79b37a.queue_time_us) (tablet-download-pool-26ba4bcaae574ccbaa5c80e81e79b37a.run_cpu_time_us, tablet-download-pool-26ba4bcaae574ccbaa5c80e81e79b37a.run_cpu_time_us) (tablet-download-pool-26ba4bcaae574ccbaa5c80e81e79b37a.run_wall_time_us, tablet-download-pool-26ba4bcaae574ccbaa5c80e81e79b37a.run_wall_time_us) (tablet-download-pool-4ccbb1e52c7d47aa8db0fe2165f98ecb.queue_time_us, tablet-download-pool-4ccbb1e52c7d47aa8db0fe2165f98ecb.queue_time_us) (tablet-download-pool-4ccbb1e52c7d47aa8db0fe2165f98ecb.run_cpu_time_us, tablet-download-pool-4ccbb1e52c7d47aa8db0fe2165f98ecb.run_cpu_time_us) (tablet-download-pool-4ccbb1e52c7d47aa8db0fe2165f98ecb.run_wall_time_us, tablet-download-pool-4ccbb1e52c7d47aa8db0fe2165f98ecb.run_wall_time_us) (tablet-download-pool-59cded2638f24227bae24907a2c1c586.queue_time_us, tablet-download-pool-59cded2638f24227bae24907a2c1c586.queue_time_us) (tablet-download-pool-59cded2638f24227bae24907a2c1c586.run_cpu_time_us, tablet-download-pool-59cded2638f24227bae24907a2c1c586.run_cpu_time_us) (tablet-download-pool-59cded2638f24227bae24907a2c1c586.run_wall_time_us, tablet-download-pool-59cded2638f24227bae24907a2c1c586.run_wall_time_us) (tablet-download-pool-7f04446f1405417ea781f0ddc8b27b68.queue_time_us, tablet-download-pool-7f04446f1405417ea781f0ddc8b27b68.queue_time_us) (tablet-download-pool-7f04446f1405417ea781f0ddc8b27b68.run_cpu_time_us, tablet-download-pool-7f04446f1405417ea781f0ddc8b27b68.run_cpu_time_us) (tablet-download-pool-7f04446f1405417ea781f0ddc8b27b68.run_wall_time_us, tablet-download-pool-7f04446f1405417ea781f0ddc8b27b68.run_wall_time_us) (tablet-download-pool-a273db7b6ae9458ca8e44c665bfb2504.queue_time_us, tablet-download-pool-a273db7b6ae9458ca8e44c665bfb2504.queue_time_us) (tablet-download-pool-a273db7b6ae9458ca8e44c665bfb2504.run_cpu_time_us, tablet-download-pool-a273db7b6ae9458ca8e44c665bfb2504.run_cpu_time_us) (tablet-download-pool-a273db7b6ae9458ca8e44c665bfb2504.run_wall_time_us, tablet-download-pool-a273db7b6ae9458ca8e44c665bfb2504.run_wall_time_us) (tablet-download-pool-a7c4443c00da46f084c2d1fd81aaf261.queue_time_us, tablet-download-pool-a7c4443c00da46f084c2d1fd81aaf261.queue_time_us) (tablet-download-pool-a7c4443c00da46f084c2d1fd81aaf261.run_cpu_time_us, tablet-download-pool-a7c4443c00da46f084c2d1fd81aaf261.run_cpu_time_us) (tablet-download-pool-a7c4443c00da46f084c2d1fd81aaf261.run_wall_time_us, tablet-download-pool-a7c4443c00da46f084c2d1fd81aaf261.run_wall_time_us) (tablet-download-pool-aa9c0fdabcfc4d10b26fd1c05043ac44.queue_time_us, tablet-download-pool-aa9c0fdabcfc4d10b26fd1c05043ac44.queue_time_us) (tablet-download-pool-aa9c0fdabcfc4d10b26fd1c05043ac44.run_cpu_time_us, tablet-download-pool-aa9c0fdabcfc4d10b26fd1c05043ac44.run_cpu_time_us) (tablet-download-pool-aa9c0fdabcfc4d10b26fd1c05043ac44.run_wall_time_us, tablet-download-pool-aa9c0fdabcfc4d10b26fd1c05043ac44.run_wall_time_us) (tablet-download-pool-c4be050ef4ec4fa8a3eb04c3fad2993b.queue_time_us, tablet-download-pool-c4be050ef4ec4fa8a3eb04c3fad2993b.queue_time_us) (tablet-download-pool-c4be050ef4ec4fa8a3eb04c3fad2993b.run_cpu_time_us, tablet-download-pool-c4be050ef4ec4fa8a3eb04c3fad2993b.run_cpu_time_us) (tablet-download-pool-c4be050ef4ec4fa8a3eb04c3fad2993b.run_wall_time_us, tablet-download-pool-c4be050ef4ec4fa8a3eb04c3fad2993b.run_wall_time_us) (tablet-download-pool-ce371c0c13444122b13ddd928e3f502f.queue_time_us, tablet-download-pool-ce371c0c13444122b13ddd928e3f502f.queue_time_us) (tablet-download-pool-eeb1cdb4349f4b62add4a02741ae7d58.queue_time_us, tablet-download-pool-eeb1cdb4349f4b62add4a02741ae7d58.queue_time_us) (tablet-download-pool-eeb1cdb4349f4b62add4a02741ae7d58.run_cpu_time_us, tablet-download-pool-eeb1cdb4349f4b62add4a02741ae7d58.run_cpu_time_us) (tablet-download-pool-eeb1cdb4349f4b62add4a02741ae7d58.run_wall_time_us, tablet-download-pool-eeb1cdb4349f4b62add4a02741ae7d58.run_wall_time_us) (tablet-download-pool-fc505438fcd54ddc8f32dd22a13390a9.queue_time_us, tablet-download-pool-fc505438fcd54ddc8f32dd22a13390a9.queue_time_us) (tablet-download-pool-fc505438fcd54ddc8f32dd22a13390a9.run_cpu_time_us, tablet-download-pool-fc505438fcd54ddc8f32dd22a13390a9.run_cpu_time_us) (tablet-download-pool-fc505438fcd54ddc8f32dd22a13390a9.run_wall_time_us, tablet-download-pool-fc505438fcd54ddc8f32dd22a13390a9.run_wall_time_us) (tablet-download-pool-fd5225972315419d89430cd3c5da941c.queue_time_us, tablet-download-pool-fd5225972315419d89430cd3c5da941c.queue_time_us) (tablet-download-pool-fd5225972315419d89430cd3c5da941c.run_cpu_time_us, tablet-download-pool-fd5225972315419d89430cd3c5da941c.run_cpu_time_us) (tablet-download-pool-fd5225972315419d89430cd3c5da941c.run_wall_time_us, tablet-download-pool-fd5225972315419d89430cd3c5da941c.run_wall_time_us) (tablet-open.queue_time_us, tablet-open.queue_time_us) (tablet-open.run_cpu_time_us, tablet-open.run_cpu_time_us) (tablet-open.run_wall_time_us, tablet-open.run_wall_time_us) (txn-commit.queue_time_us, txn-commit.queue_time_us) (txn-commit.run_cpu_time_us, txn-commit.run_cpu_time_us) (txn-commit.run_wall_time_us, txn-commit.run_wall_time_us) (txn-participant-registration.queue_time_us, txn-participant-registration.queue_time_us) (txn-participant-registration.run_cpu_time_us, txn-participant-registration.run_cpu_time_us) (txn-participant-registration.run_wall_time_us, txn-participant-registration.run_wall_time_us) (txn-status-manager.queue_time_us, txn-status-manager.queue_time_us) (txn-status-manager.run_cpu_time_us, txn-status-manager.run_cpu_time_us) (txn-status-manager.run_wall_time_us, txn-status-manager.run_wall_time_us) (txn-status-tablet-reload.queue_time_us, txn-status-tablet-reload.queue_time_us) (txn-status-tablet-reload.run_cpu_time_us, txn-status-tablet-reload.run_cpu_time_us) (txn-status-tablet-reload.run_wall_time_us, txn-status-tablet-reload.run_wall_time_us) (wal-append.queue_time_us, wal-append.queue_time_us) (wal-append.run_cpu_time_us, wal-append.run_cpu_time_us) (wal-append.run_wall_time_us, wal-append.run_wall_time_us)
*** Check failure stack trace: ***
@ 0x7f4ce7642dcd google::LogMessage::Fail() at ??:0
@ 0x7f4ce7646b93 google::LogMessage::SendToLog() at ??:0
@ 0x7f4ce76427cc google::LogMessage::Flush() at ??:0
@ 0x7f4ce7643f59 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f4cea383742 kudu::TraceMetrics::InternName() at ??:0
@ 0x7f4cea34ff8b kudu::ThreadPool::ThreadPool() at ??:0
@ 0x7f4cea349cd6 kudu::ThreadPoolBuilder::Build() at ??:0
@ 0x7f4cf979d8de kudu::tserver::TabletCopyClient::TabletCopyClient() at ??:0
@ 0x7f4cf97b5daf kudu::tserver::RemoteTabletCopyClient::RemoteTabletCopyClient() at ??:0
@ 0x7f4cf98e0480 kudu::tserver::TSTabletManager::RunTabletCopy() at ??:0
@ 0x7f4cf9918a8d kudu::tserver::TabletCopyRunnable::Run() at ??:0
@ 0x7f4cf98fb981 kudu::tserver::TSTabletManager::StartTabletCopy()::$_12::operator()() at ??:0
@ 0x7f4cf98fb70e std::_Function_handler<>::_M_invoke() at ??:0
@ 0xa60bcd std::function<>::operator()() at ??:?
@ 0x7f4cea353b75 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f4cea35719c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f4cea357019 std::_Function_handler<>::_M_invoke() at ??:0
@ 0xa60bcd std::function<>::operator()() at ??:?
@ 0x7f4cea33201e kudu::Thread::SuperviseThread() at ??:0
@ 0x7f4ced6b76db start_thread at ??:0
@ 0x7f4ce64f571f clone at ??:0
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 5 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | HEALTHY | <none> | 6 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 0 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 6 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.196:41833: Timed out: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.196:41833: received 0 of 4 requested bytes (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_server_key_size | 768 | experimental | all 6 server(s) checked
local_ip_for_outbound_sockets | 127.2.13.193 | experimental | 127.2.13.193:33069
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | all 6 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 6 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 6 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 6 server(s) checked
server_dump_info_format | pb | hidden | all 6 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.2.13.193:33069
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | all 6 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 6 server(s) checked
time_source | builtin | all 6 server(s) checked
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.197:46671, tserver@127.2.13.195:46293, and 3 other server(s)
Tablet Summary
Tablet 70d88dbe835c4ba99a8f22477926a15f of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
c120f4d1df6c4e16b63556d1f64f8a99 (127.2.13.194:37017): RUNNING
3d72f5d7124c4af5a05043929be3f350 (127.2.13.197:46671): RUNNING
40e91970c83a45debf7f4c6a20ee6d93 (127.2.13.196:41833): TS unavailable [LEADER]
All reported replicas are:
A = c120f4d1df6c4e16b63556d1f64f8a99
B = 3d72f5d7124c4af5a05043929be3f350
C = 40e91970c83a45debf7f4c6a20ee6d93
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B C* | | | Yes
A | A B C | 4 | 58 | Yes
B | A B C* | 4 | 58 | Yes
C | [config not available] | | |
Tablet fd5225972315419d89430cd3c5da941c of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
9122a9c942514b92aa63586e9538fc25 (127.2.13.198:37463): RUNNING [LEADER]
40e91970c83a45debf7f4c6a20ee6d93 (127.2.13.196:41833): TS unavailable
3d72f5d7124c4af5a05043929be3f350 (127.2.13.197:46671): RUNNING
All reported replicas are:
A = 9122a9c942514b92aa63586e9538fc25
B = 40e91970c83a45debf7f4c6a20ee6d93
C = 3d72f5d7124c4af5a05043929be3f350
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 4 | 60 | Yes
B | [config not available] | | |
C | A* B C | 4 | 60 | Yes
Tablet 523bf734b7664181a96a1ff4b5767604 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
9122a9c942514b92aa63586e9538fc25 (127.2.13.198:37463): RUNNING
40e91970c83a45debf7f4c6a20ee6d93 (127.2.13.196:41833): TS unavailable [LEADER]
3d72f5d7124c4af5a05043929be3f350 (127.2.13.197:46671): RUNNING
All reported replicas are:
A = 9122a9c942514b92aa63586e9538fc25
B = 40e91970c83a45debf7f4c6a20ee6d93
C = 3d72f5d7124c4af5a05043929be3f350
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B* C | 3 | 59 | Yes
B | [config not available] | | |
C | A B* C | 3 | 59 | Yes
Tablet b7b21c8ecf024c01a92c353394024285 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
646555016bb14084999bff39175a18f0 (127.2.13.195:46293): RUNNING
9122a9c942514b92aa63586e9538fc25 (127.2.13.198:37463): RUNNING
40e91970c83a45debf7f4c6a20ee6d93 (127.2.13.196:41833): TS unavailable [LEADER]
All reported replicas are:
A = 646555016bb14084999bff39175a18f0
B = 9122a9c942514b92aa63586e9538fc25
C = 40e91970c83a45debf7f4c6a20ee6d93
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B C* | | | Yes
A | A B C* | 4 | 60 | Yes
B | A B C* | 4 | 60 | Yes
C | [config not available] | | |
Tablet e40655007d784cdb810c61e186e98abe of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
9122a9c942514b92aa63586e9538fc25 (127.2.13.198:37463): RUNNING
40e91970c83a45debf7f4c6a20ee6d93 (127.2.13.196:41833): TS unavailable [LEADER]
3d72f5d7124c4af5a05043929be3f350 (127.2.13.197:46671): RUNNING
All reported replicas are:
A = 9122a9c942514b92aa63586e9538fc25
B = 40e91970c83a45debf7f4c6a20ee6d93
C = 3d72f5d7124c4af5a05043929be3f350
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B* C | 4 | 61 | Yes
B | [config not available] | | |
C | A B* C | 4 | 61 | Yes
Tablet 7f04446f1405417ea781f0ddc8b27b68 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
9122a9c942514b92aa63586e9538fc25 (127.2.13.198:37463): RUNNING [LEADER]
40e91970c83a45debf7f4c6a20ee6d93 (127.2.13.196:41833): TS unavailable
3d72f5d7124c4af5a05043929be3f350 (127.2.13.197:46671): RUNNING
All reported replicas are:
A = 9122a9c942514b92aa63586e9538fc25
B = 40e91970c83a45debf7f4c6a20ee6d93
C = 3d72f5d7124c4af5a05043929be3f350
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 2 | 62 | Yes
B | [config not available] | | |
C | A* B C | 2 | 62 | Yes
Tablet 0570e726d9bb426581689d5cb6d88177 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
9122a9c942514b92aa63586e9538fc25 (127.2.13.198:37463): RUNNING [LEADER]
40e91970c83a45debf7f4c6a20ee6d93 (127.2.13.196:41833): TS unavailable
3d72f5d7124c4af5a05043929be3f350 (127.2.13.197:46671): RUNNING
All reported replicas are:
A = 9122a9c942514b92aa63586e9538fc25
B = 40e91970c83a45debf7f4c6a20ee6d93
C = 3d72f5d7124c4af5a05043929be3f350
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 4 | 66 | Yes
B | [config not available] | | |
C | A* B C | 4 | 66 | Yes
Tablet ce371c0c13444122b13ddd928e3f502f of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
9122a9c942514b92aa63586e9538fc25 (127.2.13.198:37463): RUNNING [LEADER]
40e91970c83a45debf7f4c6a20ee6d93 (127.2.13.196:41833): TS unavailable
3d72f5d7124c4af5a05043929be3f350 (127.2.13.197:46671): RUNNING
All reported replicas are:
A = 9122a9c942514b92aa63586e9538fc25
B = 40e91970c83a45debf7f4c6a20ee6d93
C = 3d72f5d7124c4af5a05043929be3f350
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 4 | 60 | Yes
B | [config not available] | | |
C | A* B C | 4 | 60 | Yes
Tablet a82d80a03df24e0a825c422362effe14 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
9122a9c942514b92aa63586e9538fc25 (127.2.13.198:37463): RUNNING [LEADER]
40e91970c83a45debf7f4c6a20ee6d93 (127.2.13.196:41833): TS unavailable
3d72f5d7124c4af5a05043929be3f350 (127.2.13.197:46671): RUNNING
All reported replicas are:
A = 9122a9c942514b92aa63586e9538fc25
B = 40e91970c83a45debf7f4c6a20ee6d93
C = 3d72f5d7124c4af5a05043929be3f350
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 3 | 63 | Yes
B | [config not available] | | |
C | A* B C | 3 | 63 | Yes
Tablet 8c03af9f7884408d807c92c6cd6c0888 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
9122a9c942514b92aa63586e9538fc25 (127.2.13.198:37463): RUNNING [LEADER]
40e91970c83a45debf7f4c6a20ee6d93 (127.2.13.196:41833): TS unavailable
3d72f5d7124c4af5a05043929be3f350 (127.2.13.197:46671): RUNNING
All reported replicas are:
A = 9122a9c942514b92aa63586e9538fc25
B = 40e91970c83a45debf7f4c6a20ee6d93
C = 3d72f5d7124c4af5a05043929be3f350
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 5 | 65 | Yes
B | [config not available] | | |
C | A* B C | 5 | 65 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+------------------+---------------+---------+------------+------------------+-------------
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
RemoveReplaceInCycle | 3 | UNDER_REPLICATED | 20 | 10 | 0 | 10 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 2
First Quartile | 10
Median | 10
Third Quartile | 11
Maximum | 20
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 20
Small | c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | 2
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
Corruption: table consistency check error: 1 out of 2 table(s) are not healthy
FAILED
W20240503 08:13:45.635039 3022 meta_cache.cc:302] tablet eeb1cdb4349f4b62add4a02741ae7d58: replica 4d907d33630a4dfc8b548e94bc03f7fa (127.2.13.193:33069) has failed: Network error: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111)
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.195:46293, tserver@127.2.13.197:46671, and 3 other server(s)
Tablet Summary
Tablet 26ba4bcaae574ccbaa5c80e81e79b37a of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
3d72f5d7124c4af5a05043929be3f350 (127.2.13.197:46671): RUNNING [LEADER]
4d907d33630a4dfc8b548e94bc03f7fa (127.2.13.193:33069): TS unavailable
646555016bb14084999bff39175a18f0 (127.2.13.195:46293): RUNNING
9122a9c942514b92aa63586e9538fc25 (127.2.13.198:37463): RUNNING [NONVOTER]
All reported replicas are:
A = 3d72f5d7124c4af5a05043929be3f350
B = 4d907d33630a4dfc8b548e94bc03f7fa
C = 646555016bb14084999bff39175a18f0
D = 9122a9c942514b92aa63586e9538fc25
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C D~ | | | Yes
A | A* B C D~ | 3 | 56 | Yes
B | [config not available] | | |
C | A* B C D~ | 3 | 56 | Yes
D | A* B C D~ | 3 | 56 | Yes
Tablet 1e4f8cf1ceff4c1b805cb05eb2c57918 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
3d72f5d7124c4af5a05043929be3f350 (127.2.13.197:46671): RUNNING [LEADER]
4d907d33630a4dfc8b548e94bc03f7fa (127.2.13.193:33069): TS unavailable
c120f4d1df6c4e16b63556d1f64f8a99 (127.2.13.194:37017): RUNNING
All reported replicas are:
A = 3d72f5d7124c4af5a05043929be3f350
B = 4d907d33630a4dfc8b548e94bc03f7fa
C = c120f4d1df6c4e16b63556d1f64f8a99
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 5 | 55 | Yes
B | [config not available] | | |
C | A* B C | 5 | 55 | Yes
Tablet aa9c0fdabcfc4d10b26fd1c05043ac44 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
3d72f5d7124c4af5a05043929be3f350 (127.2.13.197:46671): RUNNING [LEADER]
4d907d33630a4dfc8b548e94bc03f7fa (127.2.13.193:33069): TS unavailable
646555016bb14084999bff39175a18f0 (127.2.13.195:46293): RUNNING
c120f4d1df6c4e16b63556d1f64f8a99 (127.2.13.194:37017): RUNNING [NONVOTER]
All reported replicas are:
A = 3d72f5d7124c4af5a05043929be3f350
B = 4d907d33630a4dfc8b548e94bc03f7fa
C = 646555016bb14084999bff39175a18f0
D = c120f4d1df6c4e16b63556d1f64f8a99
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C D~ | | | Yes
A | A* B C D~ | 1 | 54 | Yes
B | [config not available] | | |
C | A* B C D~ | 1 | 54 | Yes
D | A* B C D~ | 1 | 54 | Yes
Tablet 4ccbb1e52c7d47aa8db0fe2165f98ecb of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
3d72f5d7124c4af5a05043929be3f350 (127.2.13.197:46671): RUNNING [LEADER]
4d907d33630a4dfc8b548e94bc03f7fa (127.2.13.193:33069): TS unavailable
646555016bb14084999bff39175a18f0 (127.2.13.195:46293): RUNNING
All reported replicas are:
A = 3d72f5d7124c4af5a05043929be3f350
B = 4d907d33630a4dfc8b548e94bc03f7fa
C = 646555016bb14084999bff39175a18f0
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 3 | 55 | Yes
B | [config not available] | | |
C | A* B C | 3 | 55 | Yes
Tablet eeb1cdb4349f4b62add4a02741ae7d58 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
3d72f5d7124c4af5a05043929be3f350 (127.2.13.197:46671): RUNNING
4d907d33630a4dfc8b548e94bc03f7fa (127.2.13.193:33069): TS unavailable
646555016bb14084999bff39175a18f0 (127.2.13.195:46293): RUNNING [LEADER]
All reported replicas are:
A = 3d72f5d7124c4af5a05043929be3f350
B = 4d907d33630a4dfc8b548e94bc03f7fa
C = 646555016bb14084999bff39175a18f0
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B C* | | | Yes
A | A B C* | 6 | 54 | Yes
B | [config not available] | | |
C | A B C* | 6 | 54 | Yes
Tablet a7c4443c00da46f084c2d1fd81aaf261 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
3d72f5d7124c4af5a05043929be3f350 (127.2.13.197:46671): RUNNING
4d907d33630a4dfc8b548e94bc03f7fa (127.2.13.193:33069): TS unavailable
646555016bb14084999bff39175a18f0 (127.2.13.195:46293): RUNNING [LEADER]
All reported replicas are:
A = 3d72f5d7124c4af5a05043929be3f350
B = 4d907d33630a4dfc8b548e94bc03f7fa
C = 646555016bb14084999bff39175a18f0
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B C* | | | Yes
A | A B C* | 3 | 56 | Yes
B | [config not available] | | |
C | A B C* | 3 | 56 | Yes
Tablet 59cded2638f24227bae24907a2c1c586 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
3d72f5d7124c4af5a05043929be3f350 (127.2.13.197:46671): RUNNING [LEADER]
4d907d33630a4dfc8b548e94bc03f7fa (127.2.13.193:33069): TS unavailable
646555016bb14084999bff39175a18f0 (127.2.13.195:46293): RUNNING
c120f4d1df6c4e16b63556d1f64f8a99 (127.2.13.194:37017): RUNNING [NONVOTER]
All reported replicas are:
A = 3d72f5d7124c4af5a05043929be3f350
B = 4d907d33630a4dfc8b548e94bc03f7fa
C = 646555016bb14084999bff39175a18f0
D = c120f4d1df6c4e16b63556d1f64f8a99
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C D~ | | | Yes
A | A* B C D~ | 2 | 56 | Yes
B | [config not available] | | |
C | A* B C D~ | 2 | 56 | Yes
D | A B C D~ | 2 | 56 | Yes
Tablet fc505438fcd54ddc8f32dd22a13390a9 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
3d72f5d7124c4af5a05043929be3f350 (127.2.13.197:46671): RUNNING [LEADER]
9122a9c942514b92aa63586e9538fc25 (127.2.13.198:37463): RUNNING
4d907d33630a4dfc8b548e94bc03f7fa (127.2.13.193:33069): TS unavailable
All reported replicas are:
A = 3d72f5d7124c4af5a05043929be3f350
B = 9122a9c942514b92aa63586e9538fc25
C = 4d907d33630a4dfc8b548e94bc03f7fa
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 4 | 48 | Yes
B | A* B C | 4 | 48 | Yes
C | [config not available] | | |
Tablet a273db7b6ae9458ca8e44c665bfb2504 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
3d72f5d7124c4af5a05043929be3f350 (127.2.13.197:46671): RUNNING [LEADER]
4d907d33630a4dfc8b548e94bc03f7fa (127.2.13.193:33069): TS unavailable
646555016bb14084999bff39175a18f0 (127.2.13.195:46293): RUNNING
All reported replicas are:
A = 3d72f5d7124c4af5a05043929be3f350
B = 4d907d33630a4dfc8b548e94bc03f7fa
C = 646555016bb14084999bff39175a18f0
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 3 | 56 | Yes
B | [config not available] | | |
C | A* B C | 3 | 56 | Yes
Tablet c4be050ef4ec4fa8a3eb04c3fad2993b of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
3d72f5d7124c4af5a05043929be3f350 (127.2.13.197:46671): RUNNING [LEADER]
4d907d33630a4dfc8b548e94bc03f7fa (127.2.13.193:33069): TS unavailable
646555016bb14084999bff39175a18f0 (127.2.13.195:46293): RUNNING
All reported replicas are:
A = 3d72f5d7124c4af5a05043929be3f350
B = 4d907d33630a4dfc8b548e94bc03f7fa
C = 646555016bb14084999bff39175a18f0
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 4 | 54 | Yes
B | [config not available] | | |
C | A* B C | 4 | 54 | Yes
Tablet 0f5de6a6722740af83fa8194084edbc0 of table 'TestTable' is under-replicated: 1 replica(s) not RUNNING
3d72f5d7124c4af5a05043929be3f350 (127.2.13.197:46671): RUNNING [LEADER]
4d907d33630a4dfc8b548e94bc03f7fa (127.2.13.193:33069): TS unavailable
646555016bb14084999bff39175a18f0 (127.2.13.195:46293): RUNNING
9122a9c942514b92aa63586e9538fc25 (127.2.13.198:37463): not running [NONVOTER]
State: FAILED
Data state: TABLET_DATA_TOMBSTONED
Last status: Tablet Copy: Tombstoned tablet 0f5de6a6722740af83fa8194084edbc0: Tablet copy aborted
All reported replicas are:
A = 3d72f5d7124c4af5a05043929be3f350
B = 4d907d33630a4dfc8b548e94bc03f7fa
C = 646555016bb14084999bff39175a18f0
D = 9122a9c942514b92aa63586e9538fc25
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C D~ | | | Yes
A | A* B C D~ | 2 | 9 | Yes
B | [config not available] | | |
C | A* B C D~ | 2 | 9 | Yes
D | [config not available] | | |
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+------------------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | UNDER_REPLICATED | 20 | 10 | 0 | 10 | 0
TestTable | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 5
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 67
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
Corruption: table consistency check error: 2 out of 2 table(s) are not healthy
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.197:46671, tserver@127.2.13.196:41833, and 3 other server(s)
Tablet Summary
Tablet 1e4f8cf1ceff4c1b805cb05eb2c57918 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
3d72f5d7124c4af5a05043929be3f350 (127.2.13.197:46671): RUNNING [LEADER]
4d907d33630a4dfc8b548e94bc03f7fa (127.2.13.193:33069): TS unavailable
c120f4d1df6c4e16b63556d1f64f8a99 (127.2.13.194:37017): RUNNING
646555016bb14084999bff39175a18f0 (127.2.13.195:46293): not running [NONVOTER]
State: FAILED
Data state: TABLET_DATA_TOMBSTONED
Last status: Tablet Copy: Tombstoned tablet 1e4f8cf1ceff4c1b805cb05eb2c57918: Tablet copy aborted
All reported replicas are:
A = 3d72f5d7124c4af5a05043929be3f350
B = 4d907d33630a4dfc8b548e94bc03f7fa
C = c120f4d1df6c4e16b63556d1f64f8a99
D = 646555016bb14084999bff39175a18f0
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C D~ | | | Yes
A | A* B C D~ | 5 | 59 | Yes
B | [config not available] | | |
C | A* B C D~ | 5 | 59 | Yes
D | [config not available] | | |
Tablet eeb1cdb4349f4b62add4a02741ae7d58 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
3d72f5d7124c4af5a05043929be3f350 (127.2.13.197:46671): RUNNING
4d907d33630a4dfc8b548e94bc03f7fa (127.2.13.193:33069): TS unavailable
646555016bb14084999bff39175a18f0 (127.2.13.195:46293): RUNNING [LEADER]
c120f4d1df6c4e16b63556d1f64f8a99 (127.2.13.194:37017): RUNNING [NONVOTER]
All reported replicas are:
A = 3d72f5d7124c4af5a05043929be3f350
B = 4d907d33630a4dfc8b548e94bc03f7fa
C = 646555016bb14084999bff39175a18f0
D = c120f4d1df6c4e16b63556d1f64f8a99
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B C* D~ | | | Yes
A | A B C* D~ | 6 | 58 | Yes
B | [config not available] | | |
C | A C* D | 6 | 62 | Yes
D | A B C* D~ | 6 | 58 | Yes
Tablet a7c4443c00da46f084c2d1fd81aaf261 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
3d72f5d7124c4af5a05043929be3f350 (127.2.13.197:46671): RUNNING
4d907d33630a4dfc8b548e94bc03f7fa (127.2.13.193:33069): TS unavailable
646555016bb14084999bff39175a18f0 (127.2.13.195:46293): RUNNING [LEADER]
c120f4d1df6c4e16b63556d1f64f8a99 (127.2.13.194:37017): RUNNING [NONVOTER]
All reported replicas are:
A = 3d72f5d7124c4af5a05043929be3f350
B = 4d907d33630a4dfc8b548e94bc03f7fa
C = 646555016bb14084999bff39175a18f0
D = c120f4d1df6c4e16b63556d1f64f8a99
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B C* D~ | | | Yes
A | A B C* D~ | 3 | 60 | Yes
B | [config not available] | | |
C | A B C* D~ | 3 | 60 | Yes
D | A B C* D~ | 3 | 60 | Yes
Tablet fc505438fcd54ddc8f32dd22a13390a9 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
3d72f5d7124c4af5a05043929be3f350 (127.2.13.197:46671): RUNNING [LEADER]
9122a9c942514b92aa63586e9538fc25 (127.2.13.198:37463): RUNNING
4d907d33630a4dfc8b548e94bc03f7fa (127.2.13.193:33069): TS unavailable
c120f4d1df6c4e16b63556d1f64f8a99 (127.2.13.194:37017): RUNNING [NONVOTER]
All reported replicas are:
A = 3d72f5d7124c4af5a05043929be3f350
B = 9122a9c942514b92aa63586e9538fc25
C = 4d907d33630a4dfc8b548e94bc03f7fa
D = c120f4d1df6c4e16b63556d1f64f8a99
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C D~ | | | Yes
A | A* B C D~ | 4 | 55 | Yes
B | A* B C D~ | 4 | 55 | Yes
C | [config not available] | | |
D | A* B C D~ | 4 | 55 | Yes
Tablet a273db7b6ae9458ca8e44c665bfb2504 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
3d72f5d7124c4af5a05043929be3f350 (127.2.13.197:46671): RUNNING [LEADER]
4d907d33630a4dfc8b548e94bc03f7fa (127.2.13.193:33069): TS unavailable
646555016bb14084999bff39175a18f0 (127.2.13.195:46293): RUNNING
c120f4d1df6c4e16b63556d1f64f8a99 (127.2.13.194:37017): RUNNING [NONVOTER]
All reported replicas are:
A = 3d72f5d7124c4af5a05043929be3f350
B = 4d907d33630a4dfc8b548e94bc03f7fa
C = 646555016bb14084999bff39175a18f0
D = c120f4d1df6c4e16b63556d1f64f8a99
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C D~ | | | Yes
A | A* B C D~ | 3 | 60 | Yes
B | [config not available] | | |
C | A* B C D~ | 3 | 60 | Yes
D | A* B C D~ | 3 | 60 | Yes
Tablet c4be050ef4ec4fa8a3eb04c3fad2993b of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
3d72f5d7124c4af5a05043929be3f350 (127.2.13.197:46671): RUNNING [LEADER]
4d907d33630a4dfc8b548e94bc03f7fa (127.2.13.193:33069): TS unavailable
646555016bb14084999bff39175a18f0 (127.2.13.195:46293): RUNNING
c120f4d1df6c4e16b63556d1f64f8a99 (127.2.13.194:37017): RUNNING [NONVOTER]
All reported replicas are:
A = 3d72f5d7124c4af5a05043929be3f350
B = 4d907d33630a4dfc8b548e94bc03f7fa
C = 646555016bb14084999bff39175a18f0
D = c120f4d1df6c4e16b63556d1f64f8a99
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C D~ | | | Yes
A | A* B C D~ | 4 | 58 | Yes
B | [config not available] | | |
C | A* B C D~ | 4 | 58 | Yes
D | A* B C D~ | 4 | 58 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+------------------+---------------+---------+------------+------------------+-------------
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
RemoveReplaceInCycle | 3 | UNDER_REPLICATED | 20 | 14 | 0 | 6 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 6
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 69
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
Corruption: table consistency check error: 1 out of 2 table(s) are not healthy
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.196:41833, tserver@127.2.13.197:46671, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.197:46671, tserver@127.2.13.196:41833, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.195:46293, tserver@127.2.13.196:41833, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.198:37463, tserver@127.2.13.197:46671, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.196:41833, tserver@127.2.13.195:46293, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.198:37463, tserver@127.2.13.197:46671, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.196:41833, tserver@127.2.13.197:46671, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.197:46671, tserver@127.2.13.196:41833, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.197:46671, tserver@127.2.13.196:41833, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.196:41833, tserver@127.2.13.197:46671, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.198:37463, tserver@127.2.13.195:46293, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.196:41833, tserver@127.2.13.197:46671, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.196:41833, tserver@127.2.13.194:37017, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.194:37017, tserver@127.2.13.196:41833, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.196:41833, tserver@127.2.13.194:37017, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.196:41833, tserver@127.2.13.197:46671, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.197:46671, tserver@127.2.13.196:41833, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.195:46293, tserver@127.2.13.196:41833, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.196:41833, tserver@127.2.13.197:46671, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.197:46671, tserver@127.2.13.198:37463, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.195:46293, tserver@127.2.13.196:41833, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.197:46671, tserver@127.2.13.195:46293, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.196:41833, tserver@127.2.13.197:46671, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.197:46671, tserver@127.2.13.195:46293, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.195:46293, tserver@127.2.13.197:46671, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.197:46671, tserver@127.2.13.196:41833, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.197:46671, tserver@127.2.13.196:41833, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.197:46671, tserver@127.2.13.195:46293, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.197:46671, tserver@127.2.13.195:46293, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.196:41833, tserver@127.2.13.197:46671, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.197:46671, tserver@127.2.13.194:37017, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.196:41833, tserver@127.2.13.197:46671, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.197:46671, tserver@127.2.13.196:41833, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.197:46671, tserver@127.2.13.196:41833, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.197:46671, tserver@127.2.13.196:41833, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.197:46671, tserver@127.2.13.194:37017, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.197:46671, tserver@127.2.13.196:41833, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.197:46671, tserver@127.2.13.194:37017, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.196:41833, tserver@127.2.13.198:37463, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.196:41833, tserver@127.2.13.197:46671, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.197:46671, tserver@127.2.13.195:46293, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.196:41833, tserver@127.2.13.197:46671, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.196:41833, tserver@127.2.13.197:46671, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.196:41833, tserver@127.2.13.195:46293, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.197:46671, tserver@127.2.13.196:41833, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.196:41833, tserver@127.2.13.197:46671, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.197:46671, tserver@127.2.13.195:46293, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
49e1b27cc8724685a4d9d69234661baa | 127.2.13.254:46769 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.2.13.212:33833 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | HEALTHY | <none> | 12 | 0
40e91970c83a45debf7f4c6a20ee6d93 | 127.2.13.196:41833 | HEALTHY | <none> | 0 | 0
646555016bb14084999bff39175a18f0 | 127.2.13.195:46293 | HEALTHY | <none> | 2 | 0
9122a9c942514b92aa63586e9538fc25 | 127.2.13.198:37463 | HEALTHY | <none> | 7 | 0
c120f4d1df6c4e16b63556d1f64f8a99 | 127.2.13.194:37017 | HEALTHY | <none> | 0 | 0
4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.2.13.193:33069: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.2.13.193:33069: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
local_ip_for_outbound_sockets | 127.2.13.194 | experimental | 127.2.13.194:37017
local_ip_for_outbound_sockets | 127.2.13.195 | experimental | 127.2.13.195:46293
local_ip_for_outbound_sockets | 127.2.13.196 | experimental | 127.2.13.196:41833
local_ip_for_outbound_sockets | 127.2.13.197 | experimental | 127.2.13.197:46671
local_ip_for_outbound_sockets | 127.2.13.198 | experimental | 127.2.13.198:37463
never_fsync | true | unsafe,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.2.13.194:37017
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.2.13.195:46293
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.2.13.196:41833
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.2.13.197:46671
server_dump_info_path | /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.2.13.198:37463
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.2.13.212:33833 | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
time_source | builtin | 127.2.13.197:46671, 127.2.13.196:41833, 127.2.13.195:46293, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.2.13.254:46769, tserver@127.2.13.196:41833, tserver@127.2.13.195:46293, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 8
Median | 11
Third Quartile | 12
Maximum | 21
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------+---------------
Big | 3d72f5d7124c4af5a05043929be3f350 | 127.2.13.197:46671 | 21
Small | 4d907d33630a4dfc8b548e94bc03f7fa | 127.2.13.193:33069 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
E20240503 08:14:23.511485 2103 external_mini_cluster.cc:943] Process with UUID 4d907d33630a4dfc8b548e94bc03f7fa has crashed
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/mini-cluster/external_mini_cluster.cc:947: Failure
Expected equality of these values:
0
num_crashes
Which is: 1
At least one process crashed
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/raft_consensus_stress-itest.cc:263: Failure
Expected: cluster_->AssertNoCrashes() doesn't generate new fatal failures in the current thread.
Actual: it does.
I20240503 08:14:24.548009 2103 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksV_7T9/build/asan/bin/kudu pid 2336
I20240503 08:14:24.871037 2103 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV_7T9/build/asan/bin/kudu with pid 2336
I20240503 08:14:24.912966 2103 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksV_7T9/build/asan/bin/kudu pid 2466
I20240503 08:14:25.227135 2103 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV_7T9/build/asan/bin/kudu with pid 2466
I20240503 08:14:25.270396 2103 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksV_7T9/build/asan/bin/kudu pid 2596
I20240503 08:14:25.574970 2103 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV_7T9/build/asan/bin/kudu with pid 2596
I20240503 08:14:25.616469 2103 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksV_7T9/build/asan/bin/kudu pid 2726
I20240503 08:14:25.925644 2103 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV_7T9/build/asan/bin/kudu with pid 2726
I20240503 08:14:25.975566 2103 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksV_7T9/build/asan/bin/kudu pid 2856
I20240503 08:14:26.275033 2103 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV_7T9/build/asan/bin/kudu with pid 2856
I20240503 08:14:26.315249 2103 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasksV_7T9/build/asan/bin/kudu pid 2115
I20240503 08:14:26.570396 2103 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV_7T9/build/asan/bin/kudu with pid 2115
2024-05-03T08:14:26Z chronyd exiting
I20240503 08:14:26.622997 2103 test_util.cc:175] -----------------------------------------------
I20240503 08:14:26.623114 2103 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-tasksV_7T9/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714723972449992-2103-0
[ FAILED ] RaftConsensusStressITest.RemoveReplaceInCycle (94109 ms)
[----------] 1 test from RaftConsensusStressITest (94109 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (94109 ms total)
[ PASSED ] 0 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] RaftConsensusStressITest.RemoveReplaceInCycle
1 FAILED TEST
I20240503 08:14:26.625020 2103 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/meta_cache.cc:302: suppressed but not reported on 5 messages since previous log ~40 seconds ago