[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from RaftConsensusStressITest
[ RUN ] RaftConsensusStressITest.RemoveReplaceInCycle
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20240503 02:12:19.744261 6490 test_util.cc:268] Using random seed: 1071325651
I20240503 02:12:19.755930 6490 ts_itest-base.cc:112] Starting cluster with:
I20240503 02:12:19.756016 6490 ts_itest-base.cc:113] --------------
I20240503 02:12:19.756060 6490 ts_itest-base.cc:114] 6 tablet servers
I20240503 02:12:19.756127 6490 ts_itest-base.cc:115] 3 replicas per TS
I20240503 02:12:19.756197 6490 ts_itest-base.cc:116] --------------
2024-05-03T02:12:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-03T02:12:19Z Disabled control of system clock
I20240503 02:12:19.781693 6490 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2hRXf8/build/asan/bin/kudu
/tmp/dist-test-task2hRXf8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.86.190:38473
--webserver_interface=127.6.86.190
--webserver_port=0
--builtin_ntp_servers=127.6.86.148:36333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.86.190:38473
--minloglevel=2
--raft_prepare_replacement_before_eviction=true
--max_create_tablets_per_ts=20 with env {}
I20240503 02:12:20.208671 6490 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2hRXf8/build/asan/bin/kudu as pid 6502
I20240503 02:12:20.208905 6490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240503 02:12:20.312547 6490 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2hRXf8/build/asan/bin/kudu
/tmp/dist-test-task2hRXf8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.86.129:0
--local_ip_for_outbound_sockets=127.6.86.129
--webserver_interface=127.6.86.129
--webserver_port=0
--tserver_master_addrs=127.6.86.190:38473
--builtin_ntp_servers=127.6.86.148:36333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--minloglevel=2
--tablet_copy_early_session_timeout_prob=0.125
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=2
--consensus_rpc_timeout_ms=2000
--heartbeat_interval_ms=500
--raft_heartbeat_interval_ms=200 with env {}
I20240503 02:12:20.749501 6490 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2hRXf8/build/asan/bin/kudu as pid 6592
I20240503 02:12:20.749727 6490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240503 02:12:20.753283 6490 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2hRXf8/build/asan/bin/kudu
/tmp/dist-test-task2hRXf8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.86.130:0
--local_ip_for_outbound_sockets=127.6.86.130
--webserver_interface=127.6.86.130
--webserver_port=0
--tserver_master_addrs=127.6.86.190:38473
--builtin_ntp_servers=127.6.86.148:36333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--minloglevel=2
--tablet_copy_early_session_timeout_prob=0.125
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=2
--consensus_rpc_timeout_ms=2000
--heartbeat_interval_ms=500
--raft_heartbeat_interval_ms=200 with env {}
I20240503 02:12:21.200330 6490 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2hRXf8/build/asan/bin/kudu as pid 6723
I20240503 02:12:21.200512 6490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240503 02:12:21.204507 6490 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2hRXf8/build/asan/bin/kudu
/tmp/dist-test-task2hRXf8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.86.131:0
--local_ip_for_outbound_sockets=127.6.86.131
--webserver_interface=127.6.86.131
--webserver_port=0
--tserver_master_addrs=127.6.86.190:38473
--builtin_ntp_servers=127.6.86.148:36333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--minloglevel=2
--tablet_copy_early_session_timeout_prob=0.125
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=2
--consensus_rpc_timeout_ms=2000
--heartbeat_interval_ms=500
--raft_heartbeat_interval_ms=200 with env {}
I20240503 02:12:21.669092 6490 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2hRXf8/build/asan/bin/kudu as pid 6853
I20240503 02:12:21.669304 6490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240503 02:12:21.672544 6490 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2hRXf8/build/asan/bin/kudu
/tmp/dist-test-task2hRXf8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.86.132:0
--local_ip_for_outbound_sockets=127.6.86.132
--webserver_interface=127.6.86.132
--webserver_port=0
--tserver_master_addrs=127.6.86.190:38473
--builtin_ntp_servers=127.6.86.148:36333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--minloglevel=2
--tablet_copy_early_session_timeout_prob=0.125
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=2
--consensus_rpc_timeout_ms=2000
--heartbeat_interval_ms=500
--raft_heartbeat_interval_ms=200 with env {}
I20240503 02:12:22.130137 6490 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2hRXf8/build/asan/bin/kudu as pid 6983
I20240503 02:12:22.130334 6490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240503 02:12:22.134002 6490 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2hRXf8/build/asan/bin/kudu
/tmp/dist-test-task2hRXf8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.86.133:0
--local_ip_for_outbound_sockets=127.6.86.133
--webserver_interface=127.6.86.133
--webserver_port=0
--tserver_master_addrs=127.6.86.190:38473
--builtin_ntp_servers=127.6.86.148:36333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--minloglevel=2
--tablet_copy_early_session_timeout_prob=0.125
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=2
--consensus_rpc_timeout_ms=2000
--heartbeat_interval_ms=500
--raft_heartbeat_interval_ms=200 with env {}
I20240503 02:12:22.592038 6490 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2hRXf8/build/asan/bin/kudu as pid 7113
I20240503 02:12:22.592221 6490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240503 02:12:22.595516 6490 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2hRXf8/build/asan/bin/kudu
/tmp/dist-test-task2hRXf8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.86.134:0
--local_ip_for_outbound_sockets=127.6.86.134
--webserver_interface=127.6.86.134
--webserver_port=0
--tserver_master_addrs=127.6.86.190:38473
--builtin_ntp_servers=127.6.86.148:36333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--minloglevel=2
--tablet_copy_early_session_timeout_prob=0.125
--raft_prepare_replacement_before_eviction=true
--follower_unavailable_considered_failed_sec=2
--consensus_rpc_timeout_ms=2000
--heartbeat_interval_ms=500
--raft_heartbeat_interval_ms=200 with env {}
I20240503 02:12:23.064553 6490 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2hRXf8/build/asan/bin/kudu as pid 7243
I20240503 02:12:23.064746 6490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/wal/instance
I20240503 02:12:23.082142 6490 external_mini_cluster.cc:930] 6 TS(s) registered with all masters
I20240503 02:12:23.338722 6490 external_mini_cluster.cc:930] 6 TS(s) registered with all masters
I20240503 02:12:23.341660 6490 ts_itest-base.cc:233] Waiting for 0 tablets on tserver a6d2a3e88c7a4292b6d18a800b858a3b to finish bootstrapping
I20240503 02:12:23.351554 6490 ts_itest-base.cc:233] Waiting for 0 tablets on tserver cc40c18e77b541cd9fdd62296024232f to finish bootstrapping
I20240503 02:12:23.360394 6490 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 0d89520c6a8444d68976426bf5bd599a to finish bootstrapping
I20240503 02:12:23.368719 6490 ts_itest-base.cc:233] Waiting for 1 tablets on tserver daca0c57e7b548f9b524c7419ca3b72e to finish bootstrapping
I20240503 02:12:23.375857 6490 ts_itest-base.cc:233] Waiting for 0 tablets on tserver a253d33e5a7741a1a4297187cf0226f9 to finish bootstrapping
I20240503 02:12:23.384654 6490 ts_itest-base.cc:233] Waiting for 1 tablets on tserver a21a8a818b624b7ba41c925e5bb0eed1 to finish bootstrapping
I20240503 02:12:23.391986 6490 test_util.cc:268] Using random seed: 1074973360
W20240503 02:12:50.669692 8346 negotiation.cc:337] Failed RPC negotiation. Trace:
0503 02:12:47.636063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.86.132:33421 (local address 127.0.0.1:42996)
0503 02:12:47.636317 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0503 02:12:47.636337 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0503 02:12:47.636450 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0503 02:12:50.668191 (+3031741us) negotiation.cc:327] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.6.86.132:33421: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":117}
F20240503 02:12:52.453409 8459 trace_metrics.cc:68] Check failed: g_intern_map->size() < 100 (100 vs. 100) Too many interned strings: (MaintenanceMgr.queue_time_us, MaintenanceMgr.queue_time_us) (MaintenanceMgr.run_cpu_time_us, MaintenanceMgr.run_cpu_time_us) (MaintenanceMgr.run_wall_time_us, MaintenanceMgr.run_wall_time_us) (apply.queue_time_us, apply.queue_time_us) (apply.run_cpu_time_us, apply.run_cpu_time_us) (apply.run_wall_time_us, apply.run_wall_time_us) (client-negotiator.queue_time_us, client-negotiator.queue_time_us) (client-negotiator.run_cpu_time_us, client-negotiator.run_cpu_time_us) (client-negotiator.run_wall_time_us, client-negotiator.run_wall_time_us) (compiler_manager_pool.queue_time_us, compiler_manager_pool.queue_time_us) (compiler_manager_pool.run_cpu_time_us, compiler_manager_pool.run_cpu_time_us) (compiler_manager_pool.run_wall_time_us, compiler_manager_pool.run_wall_time_us) (dirs.queue_time_us, dirs.queue_time_us) (dirs.run_cpu_time_us, dirs.run_cpu_time_us) (dirs.run_wall_time_us, dirs.run_wall_time_us) (dns-resolver.queue_time_us, dns-resolver.queue_time_us) (dns-resolver.run_cpu_time_us, dns-resolver.run_cpu_time_us) (dns-resolver.run_wall_time_us, dns-resolver.run_wall_time_us) (log-alloc.queue_time_us, log-alloc.queue_time_us) (log-alloc.run_cpu_time_us, log-alloc.run_cpu_time_us) (log-alloc.run_wall_time_us, log-alloc.run_wall_time_us) (prepare.queue_time_us, prepare.queue_time_us) (prepare.run_cpu_time_us, prepare.run_cpu_time_us) (prepare.run_wall_time_us, prepare.run_wall_time_us) (raft.queue_time_us, raft.queue_time_us) (raft.run_cpu_time_us, raft.run_cpu_time_us) (raft.run_wall_time_us, raft.run_wall_time_us) (repair_pool.queue_time_us, repair_pool.queue_time_us) (repair_pool.run_cpu_time_us, repair_pool.run_cpu_time_us) (repair_pool.run_wall_time_us, repair_pool.run_wall_time_us) (server-negotiator.queue_time_us, server-negotiator.queue_time_us) (server-negotiator.run_cpu_time_us, server-negotiator.run_cpu_time_us) (server-negotiator.run_wall_time_us, server-negotiator.run_wall_time_us) (tablet-copy.queue_time_us, tablet-copy.queue_time_us) (tablet-copy.run_cpu_time_us, tablet-copy.run_cpu_time_us) (tablet-copy.run_wall_time_us, tablet-copy.run_wall_time_us) (tablet-delete.queue_time_us, tablet-delete.queue_time_us) (tablet-delete.run_cpu_time_us, tablet-delete.run_cpu_time_us) (tablet-delete.run_wall_time_us, tablet-delete.run_wall_time_us) (tablet-download-pool-231ba0648cf94dabaa06220957f012e5.queue_time_us, tablet-download-pool-231ba0648cf94dabaa06220957f012e5.queue_time_us) (tablet-download-pool-325dd0b080ed422d9f4795fbac802e56.queue_time_us, tablet-download-pool-325dd0b080ed422d9f4795fbac802e56.queue_time_us) (tablet-download-pool-325dd0b080ed422d9f4795fbac802e56.run_cpu_time_us, tablet-download-pool-325dd0b080ed422d9f4795fbac802e56.run_cpu_time_us) (tablet-download-pool-325dd0b080ed422d9f4795fbac802e56.run_wall_time_us, tablet-download-pool-325dd0b080ed422d9f4795fbac802e56.run_wall_time_us) (tablet-download-pool-47e70fd9f4374aa6ba7c3f224fd421e4.queue_time_us, tablet-download-pool-47e70fd9f4374aa6ba7c3f224fd421e4.queue_time_us) (tablet-download-pool-47e70fd9f4374aa6ba7c3f224fd421e4.run_cpu_time_us, tablet-download-pool-47e70fd9f4374aa6ba7c3f224fd421e4.run_cpu_time_us) (tablet-download-pool-47e70fd9f4374aa6ba7c3f224fd421e4.run_wall_time_us, tablet-download-pool-47e70fd9f4374aa6ba7c3f224fd421e4.run_wall_time_us) (tablet-download-pool-4d2f7eb69d434a2dbb9eebe9255b4d0b.queue_time_us, tablet-download-pool-4d2f7eb69d434a2dbb9eebe9255b4d0b.queue_time_us) (tablet-download-pool-4d2f7eb69d434a2dbb9eebe9255b4d0b.run_cpu_time_us, tablet-download-pool-4d2f7eb69d434a2dbb9eebe9255b4d0b.run_cpu_time_us) (tablet-download-pool-4d2f7eb69d434a2dbb9eebe9255b4d0b.run_wall_time_us, tablet-download-pool-4d2f7eb69d434a2dbb9eebe9255b4d0b.run_wall_time_us) (tablet-download-pool-712934f4e33a43cea226f3f8c3dc6d96.queue_time_us, tablet-download-pool-712934f4e33a43cea226f3f8c3dc6d96.queue_time_us) (tablet-download-pool-712934f4e33a43cea226f3f8c3dc6d96.run_cpu_time_us, tablet-download-pool-712934f4e33a43cea226f3f8c3dc6d96.run_cpu_time_us) (tablet-download-pool-712934f4e33a43cea226f3f8c3dc6d96.run_wall_time_us, tablet-download-pool-712934f4e33a43cea226f3f8c3dc6d96.run_wall_time_us) (tablet-download-pool-801ef10ca9924a11828a98fbf8d2ff2d.queue_time_us, tablet-download-pool-801ef10ca9924a11828a98fbf8d2ff2d.queue_time_us) (tablet-download-pool-801ef10ca9924a11828a98fbf8d2ff2d.run_cpu_time_us, tablet-download-pool-801ef10ca9924a11828a98fbf8d2ff2d.run_cpu_time_us) (tablet-download-pool-801ef10ca9924a11828a98fbf8d2ff2d.run_wall_time_us, tablet-download-pool-801ef10ca9924a11828a98fbf8d2ff2d.run_wall_time_us) (tablet-download-pool-81f0c77b47fb4c338be94842285e239f.queue_time_us, tablet-download-pool-81f0c77b47fb4c338be94842285e239f.queue_time_us) (tablet-download-pool-81f0c77b47fb4c338be94842285e239f.run_cpu_time_us, tablet-download-pool-81f0c77b47fb4c338be94842285e239f.run_cpu_time_us) (tablet-download-pool-81f0c77b47fb4c338be94842285e239f.run_wall_time_us, tablet-download-pool-81f0c77b47fb4c338be94842285e239f.run_wall_time_us) (tablet-download-pool-9dd09a0335894ec488a017622fb885d8.queue_time_us, tablet-download-pool-9dd09a0335894ec488a017622fb885d8.queue_time_us) (tablet-download-pool-9dd09a0335894ec488a017622fb885d8.run_cpu_time_us, tablet-download-pool-9dd09a0335894ec488a017622fb885d8.run_cpu_time_us) (tablet-download-pool-9dd09a0335894ec488a017622fb885d8.run_wall_time_us, tablet-download-pool-9dd09a0335894ec488a017622fb885d8.run_wall_time_us) (tablet-download-pool-ba29afb9bb0249178b712e48c9e2acef.queue_time_us, tablet-download-pool-ba29afb9bb0249178b712e48c9e2acef.queue_time_us) (tablet-download-pool-ba29afb9bb0249178b712e48c9e2acef.run_cpu_time_us, tablet-download-pool-ba29afb9bb0249178b712e48c9e2acef.run_cpu_time_us) (tablet-download-pool-ba29afb9bb0249178b712e48c9e2acef.run_wall_time_us, tablet-download-pool-ba29afb9bb0249178b712e48c9e2acef.run_wall_time_us) (tablet-download-pool-be3975e1540c43f5b99d61f1a85cc26a.queue_time_us, tablet-download-pool-be3975e1540c43f5b99d61f1a85cc26a.queue_time_us) (tablet-download-pool-be3975e1540c43f5b99d61f1a85cc26a.run_cpu_time_us, tablet-download-pool-be3975e1540c43f5b99d61f1a85cc26a.run_cpu_time_us) (tablet-download-pool-be3975e1540c43f5b99d61f1a85cc26a.run_wall_time_us, tablet-download-pool-be3975e1540c43f5b99d61f1a85cc26a.run_wall_time_us) (tablet-download-pool-c25dcf5e97a54ac7b5f31fddb70f6fd1.queue_time_us, tablet-download-pool-c25dcf5e97a54ac7b5f31fddb70f6fd1.queue_time_us) (tablet-download-pool-c25dcf5e97a54ac7b5f31fddb70f6fd1.run_cpu_time_us, tablet-download-pool-c25dcf5e97a54ac7b5f31fddb70f6fd1.run_cpu_time_us) (tablet-download-pool-c25dcf5e97a54ac7b5f31fddb70f6fd1.run_wall_time_us, tablet-download-pool-c25dcf5e97a54ac7b5f31fddb70f6fd1.run_wall_time_us) (tablet-download-pool-d207e5454f9e4767b8d95192b17408f5.queue_time_us, tablet-download-pool-d207e5454f9e4767b8d95192b17408f5.queue_time_us) (tablet-download-pool-d207e5454f9e4767b8d95192b17408f5.run_cpu_time_us, tablet-download-pool-d207e5454f9e4767b8d95192b17408f5.run_cpu_time_us) (tablet-download-pool-d207e5454f9e4767b8d95192b17408f5.run_wall_time_us, tablet-download-pool-d207e5454f9e4767b8d95192b17408f5.run_wall_time_us) (tablet-download-pool-d73a516f8e184fb0a5f2db606a246390.queue_time_us, tablet-download-pool-d73a516f8e184fb0a5f2db606a246390.queue_time_us) (tablet-download-pool-d73a516f8e184fb0a5f2db606a246390.run_cpu_time_us, tablet-download-pool-d73a516f8e184fb0a5f2db606a246390.run_cpu_time_us) (tablet-download-pool-d73a516f8e184fb0a5f2db606a246390.run_wall_time_us, tablet-download-pool-d73a516f8e184fb0a5f2db606a246390.run_wall_time_us) (tablet-download-pool-f0cc80833beb47e1945ce42b3e295bcb.queue_time_us, tablet-download-pool-f0cc80833beb47e1945ce42b3e295bcb.queue_time_us) (tablet-download-pool-f0cc80833beb47e1945ce42b3e295bcb.run_cpu_time_us, tablet-download-pool-f0cc80833beb47e1945ce42b3e295bcb.run_cpu_time_us) (tablet-download-pool-f0cc80833beb47e1945ce42b3e295bcb.run_wall_time_us, tablet-download-pool-f0cc80833beb47e1945ce42b3e295bcb.run_wall_time_us) (tablet-download-pool-f7aa0b6832234d6daf61a199c61514f5.queue_time_us, tablet-download-pool-f7aa0b6832234d6daf61a199c61514f5.queue_time_us) (tablet-download-pool-f7aa0b6832234d6daf61a199c61514f5.run_cpu_time_us, tablet-download-pool-f7aa0b6832234d6daf61a199c61514f5.run_cpu_time_us) (tablet-download-pool-f7aa0b6832234d6daf61a199c61514f5.run_wall_time_us, tablet-download-pool-f7aa0b6832234d6daf61a199c61514f5.run_wall_time_us) (tablet-open.queue_time_us, tablet-open.queue_time_us) (tablet-open.run_cpu_time_us, tablet-open.run_cpu_time_us) (tablet-open.run_wall_time_us, tablet-open.run_wall_time_us) (txn-commit.queue_time_us, txn-commit.queue_time_us) (txn-commit.run_cpu_time_us, txn-commit.run_cpu_time_us) (txn-commit.run_wall_time_us, txn-commit.run_wall_time_us) (txn-participant-registration.queue_time_us, txn-participant-registration.queue_time_us) (txn-participant-registration.run_cpu_time_us, txn-participant-registration.run_cpu_time_us) (txn-participant-registration.run_wall_time_us, txn-participant-registration.run_wall_time_us) (txn-status-manager.queue_time_us, txn-status-manager.queue_time_us) (txn-status-manager.run_cpu_time_us, txn-status-manager.run_cpu_time_us) (txn-status-manager.run_wall_time_us, txn-status-manager.run_wall_time_us) (txn-status-tablet-reload.queue_time_us, txn-status-tablet-reload.queue_time_us) (txn-status-tablet-reload.run_cpu_time_us, txn-status-tablet-reload.run_cpu_time_us) (txn-status-tablet-reload.run_wall_time_us, txn-status-tablet-reload.run_wall_time_us) (wal-append.queue_time_us, wal-append.queue_time_us) (wal-append.run_cpu_time_us, wal-append.run_cpu_time_us) (wal-append.run_wall_time_us, wal-append.run_wall_time_us)
*** Check failure stack trace: ***
@ 0x7f3f20d3adcd google::LogMessage::Fail() at ??:0
@ 0x7f3f20d3eb93 google::LogMessage::SendToLog() at ??:0
@ 0x7f3f20d3a7cc google::LogMessage::Flush() at ??:0
@ 0x7f3f20d3bf59 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f3f23a7b742 kudu::TraceMetrics::InternName() at ??:0
@ 0x7f3f23a47f8b kudu::ThreadPool::ThreadPool() at ??:0
@ 0x7f3f23a41cd6 kudu::ThreadPoolBuilder::Build() at ??:0
@ 0x7f3f32e958de kudu::tserver::TabletCopyClient::TabletCopyClient() at ??:0
@ 0x7f3f32eaddaf kudu::tserver::RemoteTabletCopyClient::RemoteTabletCopyClient() at ??:0
@ 0x7f3f32fd8480 kudu::tserver::TSTabletManager::RunTabletCopy() at ??:0
@ 0x7f3f33010a8d kudu::tserver::TabletCopyRunnable::Run() at ??:0
@ 0x7f3f32ff3981 kudu::tserver::TSTabletManager::StartTabletCopy()::$_12::operator()() at ??:0
@ 0x7f3f32ff370e std::_Function_handler<>::_M_invoke() at ??:0
@ 0xa60bcd std::function<>::operator()() at ??:?
@ 0x7f3f23a4bb75 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f3f23a4f19c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f3f23a4f019 std::_Function_handler<>::_M_invoke() at ??:0
@ 0xa60bcd std::function<>::operator()() at ??:?
@ 0x7f3f23a2a01e kudu::Thread::SuperviseThread() at ??:0
@ 0x7f3f26daf6db start_thread at ??:0
@ 0x7f3f1fbed71f clone at ??:0
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 6 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 8 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | HEALTHY | <none> | 1 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 0 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.132:33421: Timed out: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.132:33421: received 0 of 4 requested bytes (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_server_key_size | 768 | experimental | all 6 server(s) checked
local_ip_for_outbound_sockets | 127.6.86.129 | experimental | 127.6.86.129:33103
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | all 6 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 6 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 6 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 6 server(s) checked
server_dump_info_format | pb | hidden | all 6 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.6.86.129:33103
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | all 6 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 6 server(s) checked
time_source | builtin | all 6 server(s) checked
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.131:34443, tserver@127.6.86.134:40823, and 3 other server(s)
Tablet Summary
Tablet d0702f7c15264193bf79c8b979a27300 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
a253d33e5a7741a1a4297187cf0226f9 (127.6.86.133:34199): RUNNING [LEADER]
a21a8a818b624b7ba41c925e5bb0eed1 (127.6.86.134:40823): RUNNING
daca0c57e7b548f9b524c7419ca3b72e (127.6.86.132:33421): TS unavailable
All reported replicas are:
A = a253d33e5a7741a1a4297187cf0226f9
B = a21a8a818b624b7ba41c925e5bb0eed1
C = daca0c57e7b548f9b524c7419ca3b72e
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 1 | 51 | Yes
B | A* B C | 1 | 51 | Yes
C | [config not available] | | |
Tablet 9dd09a0335894ec488a017622fb885d8 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
daca0c57e7b548f9b524c7419ca3b72e (127.6.86.132:33421): TS unavailable [LEADER]
a21a8a818b624b7ba41c925e5bb0eed1 (127.6.86.134:40823): RUNNING
a6d2a3e88c7a4292b6d18a800b858a3b (127.6.86.129:33103): RUNNING
All reported replicas are:
A = daca0c57e7b548f9b524c7419ca3b72e
B = a21a8a818b624b7ba41c925e5bb0eed1
C = a6d2a3e88c7a4292b6d18a800b858a3b
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | [config not available] | | |
B | A* B C | 2 | 43 | Yes
C | A* B C | 2 | 43 | Yes
Tablet f0cc80833beb47e1945ce42b3e295bcb of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
a253d33e5a7741a1a4297187cf0226f9 (127.6.86.133:34199): RUNNING [LEADER]
a21a8a818b624b7ba41c925e5bb0eed1 (127.6.86.134:40823): RUNNING
daca0c57e7b548f9b524c7419ca3b72e (127.6.86.132:33421): TS unavailable
All reported replicas are:
A = a253d33e5a7741a1a4297187cf0226f9
B = a21a8a818b624b7ba41c925e5bb0eed1
C = daca0c57e7b548f9b524c7419ca3b72e
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 2 | 52 | Yes
B | A* B C | 2 | 52 | Yes
C | [config not available] | | |
Tablet 5b7b02708e7645ec9a5115d727999c00 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
daca0c57e7b548f9b524c7419ca3b72e (127.6.86.132:33421): TS unavailable [LEADER]
a253d33e5a7741a1a4297187cf0226f9 (127.6.86.133:34199): RUNNING
a21a8a818b624b7ba41c925e5bb0eed1 (127.6.86.134:40823): RUNNING
All reported replicas are:
A = daca0c57e7b548f9b524c7419ca3b72e
B = a253d33e5a7741a1a4297187cf0226f9
C = a21a8a818b624b7ba41c925e5bb0eed1
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | [config not available] | | |
B | A* B C | 1 | -1 | Yes
C | A* B C | 1 | -1 | Yes
Tablet 712934f4e33a43cea226f3f8c3dc6d96 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
a253d33e5a7741a1a4297187cf0226f9 (127.6.86.133:34199): RUNNING [LEADER]
a21a8a818b624b7ba41c925e5bb0eed1 (127.6.86.134:40823): RUNNING
daca0c57e7b548f9b524c7419ca3b72e (127.6.86.132:33421): TS unavailable
All reported replicas are:
A = a253d33e5a7741a1a4297187cf0226f9
B = a21a8a818b624b7ba41c925e5bb0eed1
C = daca0c57e7b548f9b524c7419ca3b72e
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 2 | 52 | Yes
B | A* B C | 2 | 52 | Yes
C | [config not available] | | |
Tablet 801ef10ca9924a11828a98fbf8d2ff2d of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
daca0c57e7b548f9b524c7419ca3b72e (127.6.86.132:33421): TS unavailable
a253d33e5a7741a1a4297187cf0226f9 (127.6.86.133:34199): RUNNING [LEADER]
a21a8a818b624b7ba41c925e5bb0eed1 (127.6.86.134:40823): RUNNING
All reported replicas are:
A = daca0c57e7b548f9b524c7419ca3b72e
B = a253d33e5a7741a1a4297187cf0226f9
C = a21a8a818b624b7ba41c925e5bb0eed1
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B* C | | | Yes
A | [config not available] | | |
B | A B* C | 2 | -1 | Yes
C | A B* C | 2 | -1 | Yes
Tablet be3975e1540c43f5b99d61f1a85cc26a of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
a253d33e5a7741a1a4297187cf0226f9 (127.6.86.133:34199): RUNNING
a21a8a818b624b7ba41c925e5bb0eed1 (127.6.86.134:40823): RUNNING [LEADER]
daca0c57e7b548f9b524c7419ca3b72e (127.6.86.132:33421): TS unavailable
All reported replicas are:
A = a253d33e5a7741a1a4297187cf0226f9
B = a21a8a818b624b7ba41c925e5bb0eed1
C = daca0c57e7b548f9b524c7419ca3b72e
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B* C | 3 | 53 | Yes
B | A B* C | 3 | 53 | Yes
C | [config not available] | | |
Tablet c25dcf5e97a54ac7b5f31fddb70f6fd1 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
daca0c57e7b548f9b524c7419ca3b72e (127.6.86.132:33421): TS unavailable
a253d33e5a7741a1a4297187cf0226f9 (127.6.86.133:34199): RUNNING
a21a8a818b624b7ba41c925e5bb0eed1 (127.6.86.134:40823): RUNNING [LEADER]
All reported replicas are:
A = daca0c57e7b548f9b524c7419ca3b72e
B = a253d33e5a7741a1a4297187cf0226f9
C = a21a8a818b624b7ba41c925e5bb0eed1
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B C* | | | Yes
A | [config not available] | | |
B | A B C* | 1 | -1 | Yes
C | A B C* | 1 | -1 | Yes
Tablet 325dd0b080ed422d9f4795fbac802e56 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
a253d33e5a7741a1a4297187cf0226f9 (127.6.86.133:34199): RUNNING
a21a8a818b624b7ba41c925e5bb0eed1 (127.6.86.134:40823): RUNNING [LEADER]
daca0c57e7b548f9b524c7419ca3b72e (127.6.86.132:33421): TS unavailable
All reported replicas are:
A = a253d33e5a7741a1a4297187cf0226f9
B = a21a8a818b624b7ba41c925e5bb0eed1
C = daca0c57e7b548f9b524c7419ca3b72e
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B* C | 3 | 52 | Yes
B | A B* C | 3 | 52 | Yes
C | [config not available] | | |
Tablet 4d2f7eb69d434a2dbb9eebe9255b4d0b of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
daca0c57e7b548f9b524c7419ca3b72e (127.6.86.132:33421): TS unavailable
a253d33e5a7741a1a4297187cf0226f9 (127.6.86.133:34199): RUNNING
a21a8a818b624b7ba41c925e5bb0eed1 (127.6.86.134:40823): RUNNING [LEADER]
All reported replicas are:
A = daca0c57e7b548f9b524c7419ca3b72e
B = a253d33e5a7741a1a4297187cf0226f9
C = a21a8a818b624b7ba41c925e5bb0eed1
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B C* | | | Yes
A | [config not available] | | |
B | A B C* | 1 | -1 | Yes
C | A B C* | 1 | -1 | Yes
Tablet f7aa0b6832234d6daf61a199c61514f5 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
a253d33e5a7741a1a4297187cf0226f9 (127.6.86.133:34199): RUNNING [LEADER]
a21a8a818b624b7ba41c925e5bb0eed1 (127.6.86.134:40823): RUNNING
daca0c57e7b548f9b524c7419ca3b72e (127.6.86.132:33421): TS unavailable
All reported replicas are:
A = a253d33e5a7741a1a4297187cf0226f9
B = a21a8a818b624b7ba41c925e5bb0eed1
C = daca0c57e7b548f9b524c7419ca3b72e
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 2 | 52 | Yes
B | A* B C | 2 | 52 | Yes
C | [config not available] | | |
Tablet 0cded7421858400b8a428d63c3c98412 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
daca0c57e7b548f9b524c7419ca3b72e (127.6.86.132:33421): TS unavailable [LEADER]
a253d33e5a7741a1a4297187cf0226f9 (127.6.86.133:34199): RUNNING
a21a8a818b624b7ba41c925e5bb0eed1 (127.6.86.134:40823): RUNNING
All reported replicas are:
A = daca0c57e7b548f9b524c7419ca3b72e
B = a253d33e5a7741a1a4297187cf0226f9
C = a21a8a818b624b7ba41c925e5bb0eed1
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | [config not available] | | |
B | A* B C | 1 | -1 | Yes
C | A* B C | 1 | -1 | Yes
Tablet d207e5454f9e4767b8d95192b17408f5 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
a253d33e5a7741a1a4297187cf0226f9 (127.6.86.133:34199): RUNNING [LEADER]
a21a8a818b624b7ba41c925e5bb0eed1 (127.6.86.134:40823): RUNNING
daca0c57e7b548f9b524c7419ca3b72e (127.6.86.132:33421): TS unavailable
All reported replicas are:
A = a253d33e5a7741a1a4297187cf0226f9
B = a21a8a818b624b7ba41c925e5bb0eed1
C = daca0c57e7b548f9b524c7419ca3b72e
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 2 | 51 | Yes
B | A* B C | 2 | 51 | Yes
C | [config not available] | | |
Tablet c3130cebdb90402880ac994db464deb0 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
daca0c57e7b548f9b524c7419ca3b72e (127.6.86.132:33421): TS unavailable [LEADER]
a253d33e5a7741a1a4297187cf0226f9 (127.6.86.133:34199): RUNNING
a21a8a818b624b7ba41c925e5bb0eed1 (127.6.86.134:40823): RUNNING
All reported replicas are:
A = daca0c57e7b548f9b524c7419ca3b72e
B = a253d33e5a7741a1a4297187cf0226f9
C = a21a8a818b624b7ba41c925e5bb0eed1
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | [config not available] | | |
B | A* B C | 1 | -1 | Yes
C | A* B C | 1 | -1 | Yes
Tablet ba29afb9bb0249178b712e48c9e2acef of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
a253d33e5a7741a1a4297187cf0226f9 (127.6.86.133:34199): RUNNING
a21a8a818b624b7ba41c925e5bb0eed1 (127.6.86.134:40823): RUNNING [LEADER]
daca0c57e7b548f9b524c7419ca3b72e (127.6.86.132:33421): TS unavailable
All reported replicas are:
A = a253d33e5a7741a1a4297187cf0226f9
B = a21a8a818b624b7ba41c925e5bb0eed1
C = daca0c57e7b548f9b524c7419ca3b72e
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B* C | 3 | 52 | Yes
B | A B* C | 3 | 52 | Yes
C | [config not available] | | |
Tablet 231ba0648cf94dabaa06220957f012e5 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
daca0c57e7b548f9b524c7419ca3b72e (127.6.86.132:33421): TS unavailable [LEADER]
a253d33e5a7741a1a4297187cf0226f9 (127.6.86.133:34199): RUNNING
a21a8a818b624b7ba41c925e5bb0eed1 (127.6.86.134:40823): RUNNING
All reported replicas are:
A = daca0c57e7b548f9b524c7419ca3b72e
B = a253d33e5a7741a1a4297187cf0226f9
C = a21a8a818b624b7ba41c925e5bb0eed1
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | [config not available] | | |
B | A* B C | 1 | -1 | Yes
C | A* B C | 1 | -1 | Yes
Tablet d73a516f8e184fb0a5f2db606a246390 of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
a253d33e5a7741a1a4297187cf0226f9 (127.6.86.133:34199): RUNNING
a21a8a818b624b7ba41c925e5bb0eed1 (127.6.86.134:40823): RUNNING [LEADER]
daca0c57e7b548f9b524c7419ca3b72e (127.6.86.132:33421): TS unavailable
All reported replicas are:
A = a253d33e5a7741a1a4297187cf0226f9
B = a21a8a818b624b7ba41c925e5bb0eed1
C = daca0c57e7b548f9b524c7419ca3b72e
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B* C | 3 | 54 | Yes
B | A B* C | 3 | 54 | Yes
C | [config not available] | | |
Tablet 81f0c77b47fb4c338be94842285e239f of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
daca0c57e7b548f9b524c7419ca3b72e (127.6.86.132:33421): TS unavailable
a253d33e5a7741a1a4297187cf0226f9 (127.6.86.133:34199): RUNNING [LEADER]
a21a8a818b624b7ba41c925e5bb0eed1 (127.6.86.134:40823): RUNNING
All reported replicas are:
A = daca0c57e7b548f9b524c7419ca3b72e
B = a253d33e5a7741a1a4297187cf0226f9
C = a21a8a818b624b7ba41c925e5bb0eed1
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B* C | | | Yes
A | [config not available] | | |
B | A B* C | 1 | -1 | Yes
C | A B* C | 1 | -1 | Yes
Tablet 58eee491051344ce91d793c962332b38 of table 'RemoveReplaceInCycle' is conflicted: 3 replicas' active configs disagree with the leader master's
cc40c18e77b541cd9fdd62296024232f (127.6.86.130:45499): RUNNING
0d89520c6a8444d68976426bf5bd599a (127.6.86.131:34443): RUNNING
a253d33e5a7741a1a4297187cf0226f9 (127.6.86.133:34199): RUNNING [LEADER]
a21a8a818b624b7ba41c925e5bb0eed1 (127.6.86.134:40823): not running [NONVOTER]
State: BOOTSTRAPPING
Data state: TABLET_DATA_READY
Last status: Bootstrap starting.
All reported replicas are:
A = cc40c18e77b541cd9fdd62296024232f
B = 0d89520c6a8444d68976426bf5bd599a
C = a253d33e5a7741a1a4297187cf0226f9
D = a21a8a818b624b7ba41c925e5bb0eed1
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------+--------------+--------------+------------
master | A B C* D~ | | | Yes
A | A B C* | 4 | 51 | Yes
B | A B C* D~ | 4 | 48 | Yes
C | A B C* | 4 | 51 | Yes
D | A B C D~ | 4 | 48 | Yes
Tablet c631fdcced004db99dc8b3d5908928ad of table 'RemoveReplaceInCycle' is under-replicated: 1 replica(s) not RUNNING
daca0c57e7b548f9b524c7419ca3b72e (127.6.86.132:33421): TS unavailable [LEADER]
a253d33e5a7741a1a4297187cf0226f9 (127.6.86.133:34199): RUNNING
a21a8a818b624b7ba41c925e5bb0eed1 (127.6.86.134:40823): RUNNING
All reported replicas are:
A = daca0c57e7b548f9b524c7419ca3b72e
B = a253d33e5a7741a1a4297187cf0226f9
C = a21a8a818b624b7ba41c925e5bb0eed1
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A* B C | | | Yes
A | [config not available] | | |
B | A* B C | 2 | -1 | Yes
C | A* B C | 2 | -1 | Yes
Tablet 47e70fd9f4374aa6ba7c3f224fd421e4 of table 'TestTable' is under-replicated: 1 replica(s) not RUNNING
a21a8a818b624b7ba41c925e5bb0eed1 (127.6.86.134:40823): RUNNING
daca0c57e7b548f9b524c7419ca3b72e (127.6.86.132:33421): TS unavailable
a6d2a3e88c7a4292b6d18a800b858a3b (127.6.86.129:33103): RUNNING [LEADER]
All reported replicas are:
A = a21a8a818b624b7ba41c925e5bb0eed1
B = daca0c57e7b548f9b524c7419ca3b72e
C = a6d2a3e88c7a4292b6d18a800b858a3b
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
master | A B C* | | | Yes
A | A B C* | 3 | 5 | Yes
B | [config not available] | | |
C | A B C* | 3 | 5 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+--------------------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | CONSENSUS_MISMATCH | 20 | 0 | 0 | 19 | 1
TestTable | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 64
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
Corruption: table consistency check error: 2 out of 2 table(s) are not healthy
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.134:40823, tserver@127.6.86.131:34443, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.131:34443, tserver@127.6.86.134:40823, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.131:34443, tserver@127.6.86.133:34199, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.131:34443, tserver@127.6.86.134:40823, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.131:34443, tserver@127.6.86.133:34199, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.133:34199, tserver@127.6.86.131:34443, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.131:34443, tserver@127.6.86.133:34199, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.131:34443, tserver@127.6.86.134:40823, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.134:40823, tserver@127.6.86.130:45499, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.134:40823, tserver@127.6.86.131:34443, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.131:34443, tserver@127.6.86.134:40823, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.131:34443, tserver@127.6.86.134:40823, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.134:40823, tserver@127.6.86.133:34199, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.133:34199, tserver@127.6.86.131:34443, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.131:34443, tserver@127.6.86.133:34199, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.134:40823, tserver@127.6.86.133:34199, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.131:34443, tserver@127.6.86.134:40823, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.131:34443, tserver@127.6.86.134:40823, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.131:34443, tserver@127.6.86.132:33421, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.134:40823, tserver@127.6.86.131:34443, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.131:34443, tserver@127.6.86.133:34199, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.133:34199, tserver@127.6.86.131:34443, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.131:34443, tserver@127.6.86.134:40823, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.133:34199, tserver@127.6.86.131:34443, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.131:34443, tserver@127.6.86.133:34199, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.133:34199, tserver@127.6.86.131:34443, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.131:34443, tserver@127.6.86.133:34199, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.130:45499, tserver@127.6.86.134:40823, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.133:34199, tserver@127.6.86.131:34443, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.131:34443, tserver@127.6.86.134:40823, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.134:40823, tserver@127.6.86.133:34199, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.133:34199, tserver@127.6.86.130:45499, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.134:40823, tserver@127.6.86.132:33421, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.131:34443, tserver@127.6.86.133:34199, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.131:34443, tserver@127.6.86.134:40823, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.131:34443, tserver@127.6.86.134:40823, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.131:34443, tserver@127.6.86.134:40823, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.131:34443, tserver@127.6.86.134:40823, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.134:40823, tserver@127.6.86.131:34443, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.131:34443, tserver@127.6.86.134:40823, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.131:34443, tserver@127.6.86.133:34199, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.133:34199, tserver@127.6.86.131:34443, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.133:34199, tserver@127.6.86.134:40823, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.131:34443, tserver@127.6.86.134:40823, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.131:34443, tserver@127.6.86.134:40823, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.131:34443, tserver@127.6.86.133:34199, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.134:40823, tserver@127.6.86.133:34199, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.131:34443, tserver@127.6.86.133:34199, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.133:34199, tserver@127.6.86.131:34443, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
d41bec257ebb4f6691484826a4f5410a | 127.6.86.190:38473 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.86.148:36333 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+-------------+----------+----------------+-----------------
0d89520c6a8444d68976426bf5bd599a | 127.6.86.131:34443 | HEALTHY | <none> | 0 | 0
a21a8a818b624b7ba41c925e5bb0eed1 | 127.6.86.134:40823 | HEALTHY | <none> | 10 | 0
a253d33e5a7741a1a4297187cf0226f9 | 127.6.86.133:34199 | HEALTHY | <none> | 10 | 0
cc40c18e77b541cd9fdd62296024232f | 127.6.86.130:45499 | HEALTHY | <none> | 1 | 0
daca0c57e7b548f9b524c7419ca3b72e | 127.6.86.132:33421 | HEALTHY | <none> | 0 | 0
a6d2a3e88c7a4292b6d18a800b858a3b | 127.6.86.129:33103 | UNAVAILABLE | <none> | n/a | n/a
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 6
Error from 127.6.86.129:33103: Network error: could not get status from server: Client connection negotiation failed: client connection to 127.6.86.129:33103: connect: Connection refused (error 111) (UNAVAILABLE)
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-----------------------------------------------------------------------------------
ipki_server_key_size | 768 | experimental | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
local_ip_for_outbound_sockets | 127.6.86.130 | experimental | 127.6.86.130:45499
local_ip_for_outbound_sockets | 127.6.86.131 | experimental | 127.6.86.131:34443
local_ip_for_outbound_sockets | 127.6.86.132 | experimental | 127.6.86.132:33421
local_ip_for_outbound_sockets | 127.6.86.133 | experimental | 127.6.86.133:34199
local_ip_for_outbound_sockets | 127.6.86.134 | experimental | 127.6.86.134:40823
never_fsync | true | unsafe,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
openssl_security_level_override | 0 | unsafe,hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
raft_prepare_replacement_before_eviction | true | experimental,advanced | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
rpc_server_allow_ephemeral_ports | true | unsafe | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_format | pb | hidden | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.86.130:45499
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.86.131:34443
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.86.132:33421
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.86.133:34199
server_dump_info_path | /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.86.134:40823
tablet_copy_early_session_timeout_prob | 0.125 | unsafe,runtime | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-----------------------------------------------------------------------------------
builtin_ntp_servers | 127.6.86.148:36333 | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
time_source | builtin | 127.6.86.131:34443, 127.6.86.134:40823, 127.6.86.133:34199, and 2 other server(s)
Version Summary
Version | Servers
-----------------+----------------------------------------------------------------------------------------------------------
1.18.0-SNAPSHOT | master@127.6.86.190:38473, tserver@127.6.86.131:34443, tserver@127.6.86.133:34199, and 3 other server(s)
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
----------------------+----+---------+---------------+---------+------------+------------------+-------------
RemoveReplaceInCycle | 3 | HEALTHY | 20 | 20 | 0 | 0 | 0
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 19
Third Quartile | 20
Maximum | 21
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 2
Tablets | 21
Replicas | 63
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
tserver unusual flags check error: 1 of 6 tservers were not available to retrieve unusual flags
tserver diverged flags check error: 1 of 6 tservers were not available to retrieve time_source category flags
==================
Errors:
==================
Network error: error fetching info from tablet servers: failed to gather info for all tablet servers: 1 of 6 had errors
FAILED
E20240503 02:13:39.478588 6490 external_mini_cluster.cc:943] Process with UUID a6d2a3e88c7a4292b6d18a800b858a3b has crashed
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/mini-cluster/external_mini_cluster.cc:947: Failure
Expected equality of these values:
0
num_crashes
Which is: 1
At least one process crashed
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/raft_consensus_stress-itest.cc:263: Failure
Expected: cluster_->AssertNoCrashes() doesn't generate new fatal failures in the current thread.
Actual: it does.
I20240503 02:13:40.565953 6490 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task2hRXf8/build/asan/bin/kudu pid 6723
I20240503 02:13:40.865005 6490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2hRXf8/build/asan/bin/kudu with pid 6723
I20240503 02:13:40.914516 6490 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task2hRXf8/build/asan/bin/kudu pid 6853
I20240503 02:13:41.211545 6490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2hRXf8/build/asan/bin/kudu with pid 6853
I20240503 02:13:41.257817 6490 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task2hRXf8/build/asan/bin/kudu pid 6983
I20240503 02:13:41.562170 6490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2hRXf8/build/asan/bin/kudu with pid 6983
I20240503 02:13:41.615734 6490 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task2hRXf8/build/asan/bin/kudu pid 7113
I20240503 02:13:41.948771 6490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2hRXf8/build/asan/bin/kudu with pid 7113
I20240503 02:13:42.000839 6490 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task2hRXf8/build/asan/bin/kudu pid 7243
I20240503 02:13:42.318888 6490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2hRXf8/build/asan/bin/kudu with pid 7243
I20240503 02:13:42.374051 6490 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task2hRXf8/build/asan/bin/kudu pid 6502
I20240503 02:13:42.634639 6490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2hRXf8/build/asan/bin/kudu with pid 6502
2024-05-03T02:13:42Z chronyd exiting
I20240503 02:13:42.694380 6490 test_util.cc:175] -----------------------------------------------
I20240503 02:13:42.694469 6490 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task2hRXf8/test-tmp/raft_consensus_stress-itest.0.RaftConsensusStressITest.RemoveReplaceInCycle.1714702339673210-6490-0
[ FAILED ] RaftConsensusStressITest.RemoveReplaceInCycle (82955 ms)
[----------] 1 test from RaftConsensusStressITest (82955 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (82955 ms total)
[ PASSED ] 0 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] RaftConsensusStressITest.RemoveReplaceInCycle
1 FAILED TEST