Note: This is test shard 3 of 3.
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from CatalogManagerConfigurations/MasterStressTest
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/2
2026-08-10T01:57:08Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-10T01:57:08Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-task_qhVVJ/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-task_qhVVJ/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/security.properties
2026-08-10 01:57:16: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-task_qhVVJ/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-task_qhVVJ/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260810 01:57:25.614662 22804 external_mini_cluster.cc:1372] Running /tmp/dist-test-task_qhVVJ/build/asan/bin/kudu
/tmp/dist-test-task_qhVVJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.69.62:35311
--webserver_interface=127.22.69.62
--webserver_port=0
--builtin_ntp_servers=127.22.69.20:43209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917
--hive_metastore_uris=thrift://127.0.0.1:37513
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260810 01:57:25.977550 23234 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:57:25.977952 23234 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:57:25.978091 23234 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260810 01:57:25.986603 23234 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:57:25.986732 23234 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:57:25.986867 23234 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:57:25.986956 23234 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:57:25.997202 23234 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.69.20:43209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:37513
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.69.62:35311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.69.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 10 Aug 2026 01:44:04 UTC on e7f111948823
build id 13266
ASAN enabled
I20260810 01:57:25.999125 23234 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:57:26.001214 23234 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260810 01:57:26.009713 23247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:57:26.009760 23244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:57:26.013056 23251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:57:26.013880 23234 server_base.cc:1061] running on GCE node
I20260810 01:57:26.014708 23234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:57:26.016677 23234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:57:26.017961 23234 hybrid_clock.cc:648] HybridClock initialized: now 1786327046017912 us; error 70 us; skew 500 ppm
I20260810 01:57:26.018318 23234 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 01:57:26.020663 23234 webserver.cc:533] Webserver started at http://127.22.69.62:40455/ using document root <none> and password file <none>
I20260810 01:57:26.021520 23234 fs_manager.cc:362] Metadata directory not provided
I20260810 01:57:26.021647 23234 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:57:26.022082 23234 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:57:26.024791 23234 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/data/instance:
uuid: "919313c0dee540a690f46dbc8208d20b"
format_stamp: "Formatted at 2026-08-10 01:57:26 on dist-test-slave-n326"
I20260810 01:57:26.025601 23234 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/wal/instance:
uuid: "919313c0dee540a690f46dbc8208d20b"
format_stamp: "Formatted at 2026-08-10 01:57:26 on dist-test-slave-n326"
I20260810 01:57:26.031427 23234 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 01:57:26.034965 23258 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:57:26.036427 23234 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 01:57:26.036603 23234 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/data,/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/wal
uuid: "919313c0dee540a690f46dbc8208d20b"
format_stamp: "Formatted at 2026-08-10 01:57:26 on dist-test-slave-n326"
I20260810 01:57:26.036814 23234 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:57:26.050499 23234 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260810 01:57:26.051397 23234 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:57:26.051692 23234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:57:26.072899 23234 rpc_server.cc:307] RPC server started. Bound to: 127.22.69.62:35311
I20260810 01:57:26.072924 23332 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.22.69.62:35311 every 8 connection(s)
I20260810 01:57:26.074771 23234 server_base.cc:1193] Dumped server information to /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/data/info.pb
I20260810 01:57:26.079430 23333 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:26.084280 22804 external_mini_cluster.cc:1434] Started /tmp/dist-test-task_qhVVJ/build/asan/bin/kudu as pid 23234
I20260810 01:57:26.084688 22804 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/wal/instance
I20260810 01:57:26.085369 23333 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } has no permanent_uuid. Determining permanent_uuid...
I20260810 01:57:26.087600 22804 external_mini_cluster.cc:1372] Running /tmp/dist-test-task_qhVVJ/build/asan/bin/kudu
/tmp/dist-test-task_qhVVJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.69.61:41945
--webserver_interface=127.22.69.61
--webserver_port=0
--builtin_ntp_servers=127.22.69.20:43209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917
--hive_metastore_uris=thrift://127.0.0.1:37513
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260810 01:57:26.104172 23333 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } has no permanent_uuid. Determining permanent_uuid...
W20260810 01:57:26.105268 23260 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 01:57:26.106879 23333 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
I20260810 01:57:26.156276 23333 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } attempt: 1
W20260810 01:57:26.158417 23333 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
I20260810 01:57:26.226730 23333 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } attempt: 2
W20260810 01:57:26.229146 23333 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
I20260810 01:57:26.320466 23333 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } attempt: 3
W20260810 01:57:26.322759 23333 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 01:57:26.427950 23339 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:57:26.428335 23339 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:57:26.428476 23339 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260810 01:57:26.436815 23339 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:57:26.436945 23339 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:57:26.437073 23339 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:57:26.437160 23339 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:57:26.447274 23339 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.69.20:43209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:37513
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.69.61:41945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.69.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 10 Aug 2026 01:44:04 UTC on e7f111948823
build id 13266
ASAN enabled
I20260810 01:57:26.448969 23339 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:57:26.450803 23339 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260810 01:57:26.458873 23348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:57:26.458914 23351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:57:26.459765 23353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:57:26.460587 23339 server_base.cc:1061] running on GCE node
I20260810 01:57:26.461308 23339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:57:26.462550 23339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:57:26.463781 23339 hybrid_clock.cc:648] HybridClock initialized: now 1786327046463742 us; error 50 us; skew 500 ppm
I20260810 01:57:26.464128 23339 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 01:57:26.466040 23333 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } attempt: 4
I20260810 01:57:26.466328 23339 webserver.cc:533] Webserver started at http://127.22.69.61:33591/ using document root <none> and password file <none>
I20260810 01:57:26.467036 23339 fs_manager.cc:362] Metadata directory not provided
I20260810 01:57:26.467227 23339 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:57:26.467607 23339 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
W20260810 01:57:26.467935 23333 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
I20260810 01:57:26.469990 23339 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/data/instance:
uuid: "b1422349180c446db6c1768b3025253c"
format_stamp: "Formatted at 2026-08-10 01:57:26 on dist-test-slave-n326"
I20260810 01:57:26.470711 23339 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/wal/instance:
uuid: "b1422349180c446db6c1768b3025253c"
format_stamp: "Formatted at 2026-08-10 01:57:26 on dist-test-slave-n326"
I20260810 01:57:26.475742 23339 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 01:57:26.478901 23365 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:57:26.480209 23339 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 01:57:26.480350 23339 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/data,/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/wal
uuid: "b1422349180c446db6c1768b3025253c"
format_stamp: "Formatted at 2026-08-10 01:57:26 on dist-test-slave-n326"
I20260810 01:57:26.480531 23339 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:57:26.507863 23339 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260810 01:57:26.508700 23339 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:57:26.508931 23339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:57:26.529238 23339 rpc_server.cc:307] RPC server started. Bound to: 127.22.69.61:41945
I20260810 01:57:26.529270 23438 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.22.69.61:41945 every 8 connection(s)
I20260810 01:57:26.530733 23339 server_base.cc:1193] Dumped server information to /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/data/info.pb
I20260810 01:57:26.534866 23439 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:26.536340 22804 external_mini_cluster.cc:1434] Started /tmp/dist-test-task_qhVVJ/build/asan/bin/kudu as pid 23339
I20260810 01:57:26.536581 22804 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/wal/instance
I20260810 01:57:26.537981 22804 external_mini_cluster.cc:1372] Running /tmp/dist-test-task_qhVVJ/build/asan/bin/kudu
/tmp/dist-test-task_qhVVJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.69.60:36917
--webserver_interface=127.22.69.60
--webserver_port=0
--builtin_ntp_servers=127.22.69.20:43209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917
--hive_metastore_uris=thrift://127.0.0.1:37513
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260810 01:57:26.543403 23439 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } has no permanent_uuid. Determining permanent_uuid...
I20260810 01:57:26.562171 23439 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } has no permanent_uuid. Determining permanent_uuid...
I20260810 01:57:26.571310 23439 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } has no permanent_uuid. Determining permanent_uuid...
W20260810 01:57:26.572351 23368 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
W20260810 01:57:26.573971 23439 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.22.69.60:36917: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:57:26.623266 23439 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } attempt: 1
W20260810 01:57:26.625420 23439 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.22.69.60:36917: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:57:26.693674 23439 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } attempt: 2
W20260810 01:57:26.696122 23439 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.22.69.60:36917: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:57:26.767215 23333 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } attempt: 5
I20260810 01:57:26.774045 23333 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } has no permanent_uuid. Determining permanent_uuid...
W20260810 01:57:26.776141 23333 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.22.69.60:36917: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:57:26.787376 23439 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } attempt: 3
W20260810 01:57:26.789354 23439 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.22.69.60:36917: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:57:26.827411 23333 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } attempt: 1
W20260810 01:57:26.829420 23333 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.22.69.60:36917: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
W20260810 01:57:26.884063 23441 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:57:26.884357 23441 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:57:26.884424 23441 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260810 01:57:26.893344 23441 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:57:26.893427 23441 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:57:26.893482 23441 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:57:26.893545 23441 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:57:26.897687 23333 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } attempt: 2
W20260810 01:57:26.899672 23333 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.22.69.60:36917: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:57:26.903892 23441 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.69.20:43209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:37513
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.69.60:36917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.69.60
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 10 Aug 2026 01:44:04 UTC on e7f111948823
build id 13266
ASAN enabled
I20260810 01:57:26.905398 23441 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:57:26.907192 23441 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260810 01:57:26.915184 23454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:57:26.915767 23456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:57:26.916126 23452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:57:26.916469 23441 server_base.cc:1061] running on GCE node
I20260810 01:57:26.917189 23441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:57:26.918432 23441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:57:26.919636 23441 hybrid_clock.cc:648] HybridClock initialized: now 1786327046919582 us; error 35 us; skew 500 ppm
I20260810 01:57:26.919978 23441 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 01:57:26.922008 23441 webserver.cc:533] Webserver started at http://127.22.69.60:44467/ using document root <none> and password file <none>
I20260810 01:57:26.922712 23441 fs_manager.cc:362] Metadata directory not provided
I20260810 01:57:26.922832 23441 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:57:26.923262 23441 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:57:26.925558 23441 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/data/instance:
uuid: "757bb3f336084510845ed58df264c721"
format_stamp: "Formatted at 2026-08-10 01:57:26 on dist-test-slave-n326"
I20260810 01:57:26.926333 23441 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/wal/instance:
uuid: "757bb3f336084510845ed58df264c721"
format_stamp: "Formatted at 2026-08-10 01:57:26 on dist-test-slave-n326"
I20260810 01:57:26.931459 23441 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260810 01:57:26.932610 23439 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } attempt: 4
W20260810 01:57:26.934671 23439 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.22.69.60:36917: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:57:26.935256 23466 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:57:26.936545 23441 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 01:57:26.936724 23441 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/data,/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/wal
uuid: "757bb3f336084510845ed58df264c721"
format_stamp: "Formatted at 2026-08-10 01:57:26 on dist-test-slave-n326"
I20260810 01:57:26.936946 23441 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:57:26.964290 23441 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260810 01:57:26.965226 23441 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:57:26.965551 23441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:57:26.989383 23441 rpc_server.cc:307] RPC server started. Bound to: 127.22.69.60:36917
I20260810 01:57:26.989400 23546 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.22.69.60:36917 every 8 connection(s)
I20260810 01:57:26.991178 23441 server_base.cc:1193] Dumped server information to /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/data/info.pb
I20260810 01:57:26.995317 23547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:27.000310 22804 external_mini_cluster.cc:1434] Started /tmp/dist-test-task_qhVVJ/build/asan/bin/kudu as pid 23441
I20260810 01:57:27.000514 22804 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/wal/instance
I20260810 01:57:27.001150 23547 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } has no permanent_uuid. Determining permanent_uuid...
I20260810 01:57:27.006351 23333 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } attempt: 3
I20260810 01:57:27.018857 23547 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } has no permanent_uuid. Determining permanent_uuid...
I20260810 01:57:27.026739 23547 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } has no permanent_uuid. Determining permanent_uuid...
I20260810 01:57:27.027123 23333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b: Bootstrap starting.
I20260810 01:57:27.031596 23333 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:27.033064 23333 log.cc:826] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b: Log is configured to *not* fsync() on all Append() calls
I20260810 01:57:27.039755 23547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721: Bootstrap starting.
I20260810 01:57:27.041260 23333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b: No bootstrap required, opened a new log
I20260810 01:57:27.043942 23547 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:27.045023 23547 log.cc:826] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721: Log is configured to *not* fsync() on all Append() calls
I20260810 01:57:27.047885 23547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721: No bootstrap required, opened a new log
I20260810 01:57:27.047974 23333 raft_consensus.cc:359] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:27.048257 23333 raft_consensus.cc:385] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:27.048386 23333 raft_consensus.cc:740] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 919313c0dee540a690f46dbc8208d20b, State: Initialized, Role: FOLLOWER
I20260810 01:57:27.048957 23333 consensus_queue.cc:260] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:27.049983 23561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:27.050287 23561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:57:27.051149 23333 sys_catalog.cc:565] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:57:27.053088 23547 raft_consensus.cc:359] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:27.053360 23547 raft_consensus.cc:385] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:27.053509 23547 raft_consensus.cc:740] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 757bb3f336084510845ed58df264c721, State: Initialized, Role: FOLLOWER
I20260810 01:57:27.054194 23547 consensus_queue.cc:260] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
W20260810 01:57:27.054880 23268 tablet.cc:2406] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 01:57:27.060189 23569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:27.060595 23569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:57:27.061534 23547 sys_catalog.cc:565] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 01:57:27.074481 23586 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 01:57:27.074777 23586 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 01:57:27.071957 23234 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: OK, instance_id { permanent_uuid: "b1422349180c446db6c1768b3025253c" instance_seqno: 1786327046523879 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 01:57:27.083360 23234 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 01:57:27.107511 23441 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: OK, instance_id { permanent_uuid: "b1422349180c446db6c1768b3025253c" instance_seqno: 1786327046523879 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 01:57:27.118999 23441 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 01:57:27.144757 23606 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 01:57:27.145008 23606 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260810 01:57:27.217820 23474 tablet.cc:2406] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 01:57:27.223515 23561 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:57:27.223812 23561 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:27.225293 23569 raft_consensus.cc:493] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:57:27.225536 23569 raft_consensus.cc:515] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:27.225679 23561 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
I20260810 01:57:27.227218 23569 leader_election.cc:290] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), b1422349180c446db6c1768b3025253c (127.22.69.61:41945)
I20260810 01:57:27.227541 23506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "919313c0dee540a690f46dbc8208d20b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "757bb3f336084510845ed58df264c721" is_pre_election: true
I20260810 01:57:27.228018 23506 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 919313c0dee540a690f46dbc8208d20b in term 0.
I20260810 01:57:27.228828 23260 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 757bb3f336084510845ed58df264c721, 919313c0dee540a690f46dbc8208d20b; no voters:
I20260810 01:57:27.228914 23302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "757bb3f336084510845ed58df264c721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "919313c0dee540a690f46dbc8208d20b" is_pre_election: true
I20260810 01:57:27.229259 23302 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 757bb3f336084510845ed58df264c721 in term 0.
I20260810 01:57:27.229425 23561 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 01:57:27.229574 23561 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 01:57:27.229674 23561 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:27.229926 23472 leader_election.cc:304] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 757bb3f336084510845ed58df264c721, 919313c0dee540a690f46dbc8208d20b; no voters:
I20260810 01:57:27.230453 23569 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 01:57:27.230602 23569 raft_consensus.cc:493] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 01:57:27.230710 23569 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:27.233251 23561 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:27.233914 23439 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } attempt: 5
I20260810 01:57:27.235553 23569 raft_consensus.cc:515] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:27.240085 23302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "757bb3f336084510845ed58df264c721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "919313c0dee540a690f46dbc8208d20b"
I20260810 01:57:27.240428 23302 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 757bb3f336084510845ed58df264c721 in current term 1: Already voted for candidate 919313c0dee540a690f46dbc8208d20b in this term.
I20260810 01:57:27.242451 23569 leader_election.cc:290] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [CANDIDATE]: Term 1 election: Requested vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), b1422349180c446db6c1768b3025253c (127.22.69.61:41945)
I20260810 01:57:27.244089 23561 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 1 election: Requested vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
I20260810 01:57:27.244645 23506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "919313c0dee540a690f46dbc8208d20b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "757bb3f336084510845ed58df264c721"
I20260810 01:57:27.244939 23506 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 919313c0dee540a690f46dbc8208d20b in current term 1: Already voted for candidate 757bb3f336084510845ed58df264c721 in this term.
I20260810 01:57:27.251262 23439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c: Bootstrap starting.
I20260810 01:57:27.255525 23439 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:27.256920 23439 log.cc:826] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c: Log is configured to *not* fsync() on all Append() calls
I20260810 01:57:27.260923 23439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c: No bootstrap required, opened a new log
I20260810 01:57:27.268646 23439 raft_consensus.cc:359] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:27.268966 23439 raft_consensus.cc:385] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:27.269094 23439 raft_consensus.cc:740] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1422349180c446db6c1768b3025253c, State: Initialized, Role: FOLLOWER
I20260810 01:57:27.269843 23439 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:27.271937 23611 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:27.272786 23439 sys_catalog.cc:565] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:57:27.272830 23400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "757bb3f336084510845ed58df264c721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1422349180c446db6c1768b3025253c"
I20260810 01:57:27.273270 23400 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:27.276896 23400 raft_consensus.cc:2468] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 757bb3f336084510845ed58df264c721 in term 1.
I20260810 01:57:27.277334 23402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "757bb3f336084510845ed58df264c721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1422349180c446db6c1768b3025253c" is_pre_election: true
I20260810 01:57:27.277617 23402 raft_consensus.cc:2376] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 757bb3f336084510845ed58df264c721 in term 1. Re-sending same reply.
I20260810 01:57:27.277860 23401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "919313c0dee540a690f46dbc8208d20b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1422349180c446db6c1768b3025253c"
I20260810 01:57:27.278251 23401 raft_consensus.cc:2393] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 919313c0dee540a690f46dbc8208d20b in current term 1: Already voted for candidate 757bb3f336084510845ed58df264c721 in this term.
I20260810 01:57:27.278579 23404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "919313c0dee540a690f46dbc8208d20b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1422349180c446db6c1768b3025253c" is_pre_election: true
I20260810 01:57:27.278868 23404 raft_consensus.cc:2393] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 919313c0dee540a690f46dbc8208d20b in current term 1: Already voted for candidate 757bb3f336084510845ed58df264c721 in this term.
I20260810 01:57:27.279287 23468 leader_election.cc:304] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c; no voters: 919313c0dee540a690f46dbc8208d20b
I20260810 01:57:27.280161 23569 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:57:27.280726 23260 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 01:57:27.281563 23561 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260810 01:57:27.288029 23611 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:57:27.294611 23569 raft_consensus.cc:697] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 1 LEADER]: Becoming Leader. State: Replica: 757bb3f336084510845ed58df264c721, State: Running, Role: LEADER
I20260810 01:57:27.295244 23569 consensus_queue.cc:237] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:27.308475 23614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 757bb3f336084510845ed58df264c721. Latest consensus state: current_term: 1 leader_uuid: "757bb3f336084510845ed58df264c721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:27.308871 23614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [sys.catalog]: This master's current role is: LEADER
I20260810 01:57:27.316653 23623 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260810 01:57:27.319155 23623 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260810 01:57:27.337611 23400 raft_consensus.cc:1275] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 1 FOLLOWER]: Refusing update from remote peer 757bb3f336084510845ed58df264c721: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 01:57:27.338619 23614 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:57:27.340585 23302 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 1 FOLLOWER]: Refusing update from remote peer 757bb3f336084510845ed58df264c721: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 01:57:27.342149 23614 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [LEADER]: Connected to new peer: Peer: permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:57:27.363238 23561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 757bb3f336084510845ed58df264c721. Latest consensus state: current_term: 1 leader_uuid: "757bb3f336084510845ed58df264c721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:27.363519 23561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:57:27.374009 23611 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 757bb3f336084510845ed58df264c721. Latest consensus state: current_term: 1 leader_uuid: "757bb3f336084510845ed58df264c721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:27.374248 23611 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:57:27.375480 23561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "757bb3f336084510845ed58df264c721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:27.375713 23561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:57:27.391876 23569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "757bb3f336084510845ed58df264c721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:27.392233 23569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [sys.catalog]: This master's current role is: LEADER
I20260810 01:57:27.397226 23614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "757bb3f336084510845ed58df264c721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:27.397532 23614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [sys.catalog]: This master's current role is: LEADER
I20260810 01:57:27.399870 23623 catalog_manager.cc:1383] Generated new cluster ID: dfa48c56b63b4d0cbf3d6124921e8a17
I20260810 01:57:27.400002 23623 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260810 01:57:27.419780 23611 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "757bb3f336084510845ed58df264c721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:27.420055 23611 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:57:27.431645 23623 catalog_manager.cc:1406] Generated new certificate authority record
I20260810 01:57:27.433621 23623 catalog_manager.cc:1540] Loading token signing keys...
W20260810 01:57:27.455991 23650 catalog_manager.cc:1594] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 01:57:27.456190 23650 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 01:57:27.473397 23623 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721: Generated new TSK 0
I20260810 01:57:27.474960 23623 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260810 01:57:27.476501 23623 catalog_manager.cc:1563] Loading latest processed Hive Metastore notification log event ID...
I20260810 01:57:27.477867 23623 catalog_manager.cc:5941] Last processed Hive Metastore notification event ID: -1
I20260810 01:57:27.496536 22804 external_mini_cluster.cc:1372] Running /tmp/dist-test-task_qhVVJ/build/asan/bin/kudu
/tmp/dist-test-task_qhVVJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.69.1:0
--local_ip_for_outbound_sockets=127.22.69.1
--webserver_interface=127.22.69.1
--webserver_port=0
--tserver_master_addrs=127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917
--builtin_ntp_servers=127.22.69.20:43209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260810 01:57:27.974786 23657 raft_consensus.cc:493] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 757bb3f336084510845ed58df264c721)
I20260810 01:57:27.975121 23657 raft_consensus.cc:515] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:27.977147 23657 leader_election.cc:290] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
I20260810 01:57:27.977705 23506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b1422349180c446db6c1768b3025253c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "757bb3f336084510845ed58df264c721" is_pre_election: true
I20260810 01:57:27.978224 23302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b1422349180c446db6c1768b3025253c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "919313c0dee540a690f46dbc8208d20b" is_pre_election: true
I20260810 01:57:27.978868 23375 leader_election.cc:304] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1422349180c446db6c1768b3025253c; no voters: 757bb3f336084510845ed58df264c721, 919313c0dee540a690f46dbc8208d20b
I20260810 01:57:27.979363 23657 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 01:57:28.012785 23655 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:57:28.013132 23655 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:57:28.013289 23655 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260810 01:57:28.021946 23655 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:57:28.022159 23655 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.69.1
I20260810 01:57:28.034395 23655 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.69.20:43209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.69.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.69.1
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.69.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 10 Aug 2026 01:44:04 UTC on e7f111948823
build id 13266
ASAN enabled
I20260810 01:57:28.036258 23655 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:57:28.038287 23655 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260810 01:57:28.048691 23664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:57:28.048843 23666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:57:28.049572 23668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:57:28.050800 23655 server_base.cc:1061] running on GCE node
I20260810 01:57:28.051441 23655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:57:28.052371 23655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:57:28.053598 23655 hybrid_clock.cc:648] HybridClock initialized: now 1786327048053553 us; error 50 us; skew 500 ppm
I20260810 01:57:28.053901 23655 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 01:57:28.056332 23655 webserver.cc:533] Webserver started at http://127.22.69.1:46865/ using document root <none> and password file <none>
I20260810 01:57:28.057104 23655 fs_manager.cc:362] Metadata directory not provided
I20260810 01:57:28.057250 23655 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:57:28.057624 23655 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:57:28.060117 23655 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-0/data/instance:
uuid: "50d44445234b4931aaa28141cdaa67ab"
format_stamp: "Formatted at 2026-08-10 01:57:28 on dist-test-slave-n326"
I20260810 01:57:28.060885 23655 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-0/wal/instance:
uuid: "50d44445234b4931aaa28141cdaa67ab"
format_stamp: "Formatted at 2026-08-10 01:57:28 on dist-test-slave-n326"
I20260810 01:57:28.066360 23655 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260810 01:57:28.069797 23678 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:57:28.071283 23655 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 01:57:28.071444 23655 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-0/data,/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-0/wal
uuid: "50d44445234b4931aaa28141cdaa67ab"
format_stamp: "Formatted at 2026-08-10 01:57:28 on dist-test-slave-n326"
I20260810 01:57:28.071678 23655 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:57:28.077453 23586 catalog_manager.cc:1295] Loaded cluster ID: dfa48c56b63b4d0cbf3d6124921e8a17
I20260810 01:57:28.077560 23586 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b: loading cluster ID for follower catalog manager: success
I20260810 01:57:28.081944 23586 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b: acquiring CA information for follower catalog manager: success
I20260810 01:57:28.084295 23586 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 01:57:28.103156 23655 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260810 01:57:28.104674 23655 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:57:28.104998 23655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:57:28.106429 23655 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260810 01:57:28.109154 23655 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:57:28.109278 23655 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:57:28.109390 23655 ts_tablet_manager.cc:616] Registered 0 tablets
I20260810 01:57:28.109534 23655 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:57:28.188431 23655 rpc_server.cc:307] RPC server started. Bound to: 127.22.69.1:45491
I20260810 01:57:28.188488 23840 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.22.69.1:45491 every 8 connection(s)
I20260810 01:57:28.190541 23655 server_base.cc:1193] Dumped server information to /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-0/data/info.pb
I20260810 01:57:28.195900 22804 external_mini_cluster.cc:1434] Started /tmp/dist-test-task_qhVVJ/build/asan/bin/kudu as pid 23655
I20260810 01:57:28.196260 22804 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-0/wal/instance
I20260810 01:57:28.205329 22804 external_mini_cluster.cc:1372] Running /tmp/dist-test-task_qhVVJ/build/asan/bin/kudu
/tmp/dist-test-task_qhVVJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.69.2:0
--local_ip_for_outbound_sockets=127.22.69.2
--webserver_interface=127.22.69.2
--webserver_port=0
--tserver_master_addrs=127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917
--builtin_ntp_servers=127.22.69.20:43209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260810 01:57:28.248420 23841 heartbeater.cc:344] Connected to a master server at 127.22.69.60:36917
I20260810 01:57:28.248981 23841 heartbeater.cc:461] Registering TS with master...
I20260810 01:57:28.250854 23841 heartbeater.cc:507] Master 127.22.69.60:36917 requested a full tablet report, sending...
I20260810 01:57:28.254770 23491 ts_manager.cc:194] Registered new tserver with Master: 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491)
I20260810 01:57:28.258389 23491 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.22.69.1:58309
I20260810 01:57:28.275502 23846 heartbeater.cc:344] Connected to a master server at 127.22.69.61:41945
I20260810 01:57:28.275707 23846 heartbeater.cc:461] Registering TS with master...
I20260810 01:57:28.276065 23846 heartbeater.cc:507] Master 127.22.69.61:41945 requested a full tablet report, sending...
I20260810 01:57:28.277930 23392 ts_manager.cc:194] Registered new tserver with Master: 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491)
I20260810 01:57:28.278522 23842 heartbeater.cc:344] Connected to a master server at 127.22.69.62:35311
I20260810 01:57:28.278698 23842 heartbeater.cc:461] Registering TS with master...
I20260810 01:57:28.279145 23842 heartbeater.cc:507] Master 127.22.69.62:35311 requested a full tablet report, sending...
I20260810 01:57:28.280745 23284 ts_manager.cc:194] Registered new tserver with Master: 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491)
I20260810 01:57:28.407739 23853 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 757bb3f336084510845ed58df264c721)
I20260810 01:57:28.407986 23853 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:28.408926 23853 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
I20260810 01:57:28.409612 23404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "919313c0dee540a690f46dbc8208d20b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1422349180c446db6c1768b3025253c" is_pre_election: true
I20260810 01:57:28.409765 23506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "919313c0dee540a690f46dbc8208d20b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "757bb3f336084510845ed58df264c721" is_pre_election: true
I20260810 01:57:28.410547 23260 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 01:57:28.410845 23853 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 01:57:28.458063 23650 catalog_manager.cc:1295] Loaded cluster ID: dfa48c56b63b4d0cbf3d6124921e8a17
I20260810 01:57:28.458165 23650 catalog_manager.cc:1588] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c: loading cluster ID for follower catalog manager: success
I20260810 01:57:28.460943 23650 catalog_manager.cc:1610] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c: acquiring CA information for follower catalog manager: success
I20260810 01:57:28.463333 23650 catalog_manager.cc:1638] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 01:57:28.497993 23856 raft_consensus.cc:493] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 757bb3f336084510845ed58df264c721)
I20260810 01:57:28.498234 23856 raft_consensus.cc:515] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:28.498989 23856 leader_election.cc:290] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
I20260810 01:57:28.499711 23302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b1422349180c446db6c1768b3025253c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "919313c0dee540a690f46dbc8208d20b" is_pre_election: true
I20260810 01:57:28.499797 23506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b1422349180c446db6c1768b3025253c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "757bb3f336084510845ed58df264c721" is_pre_election: true
I20260810 01:57:28.500506 23375 leader_election.cc:304] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1422349180c446db6c1768b3025253c; no voters: 757bb3f336084510845ed58df264c721, 919313c0dee540a690f46dbc8208d20b
I20260810 01:57:28.500808 23856 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 01:57:28.637786 23845 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:57:28.638314 23845 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:57:28.638506 23845 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260810 01:57:28.648473 23845 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:57:28.648725 23845 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.69.2
I20260810 01:57:28.660820 23845 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.69.20:43209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.69.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.69.2
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.69.2
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 10 Aug 2026 01:44:04 UTC on e7f111948823
build id 13266
ASAN enabled
I20260810 01:57:28.662843 23845 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:57:28.664914 23845 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260810 01:57:28.675743 23867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:57:28.676203 23871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:57:28.675644 23868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:57:28.678004 23845 server_base.cc:1061] running on GCE node
I20260810 01:57:28.678623 23845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:57:28.679628 23845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:57:28.680859 23845 hybrid_clock.cc:648] HybridClock initialized: now 1786327048680801 us; error 47 us; skew 500 ppm
I20260810 01:57:28.681171 23845 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 01:57:28.683342 23845 webserver.cc:533] Webserver started at http://127.22.69.2:43883/ using document root <none> and password file <none>
I20260810 01:57:28.684094 23845 fs_manager.cc:362] Metadata directory not provided
I20260810 01:57:28.684237 23845 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:57:28.684626 23845 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:57:28.687664 23845 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-1/data/instance:
uuid: "48e33bc1e8e94d4887bb64214d0b6c36"
format_stamp: "Formatted at 2026-08-10 01:57:28 on dist-test-slave-n326"
I20260810 01:57:28.688522 23845 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-1/wal/instance:
uuid: "48e33bc1e8e94d4887bb64214d0b6c36"
format_stamp: "Formatted at 2026-08-10 01:57:28 on dist-test-slave-n326"
I20260810 01:57:28.693647 23845 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260810 01:57:28.696930 23888 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:57:28.698153 23845 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 01:57:28.698323 23845 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-1/data,/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-1/wal
uuid: "48e33bc1e8e94d4887bb64214d0b6c36"
format_stamp: "Formatted at 2026-08-10 01:57:28 on dist-test-slave-n326"
I20260810 01:57:28.698575 23845 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:57:28.733590 23845 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260810 01:57:28.734637 23845 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:57:28.734968 23845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:57:28.736367 23845 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260810 01:57:28.738615 23845 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:57:28.738761 23845 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:57:28.738927 23845 ts_tablet_manager.cc:616] Registered 0 tablets
I20260810 01:57:28.739032 23845 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:57:28.764358 23841 heartbeater.cc:499] Master 127.22.69.60:36917 was elected leader, sending a full tablet report...
I20260810 01:57:28.790390 23845 rpc_server.cc:307] RPC server started. Bound to: 127.22.69.2:46271
I20260810 01:57:28.790443 24062 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.22.69.2:46271 every 8 connection(s)
I20260810 01:57:28.792277 23845 server_base.cc:1193] Dumped server information to /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-1/data/info.pb
I20260810 01:57:28.798245 22804 external_mini_cluster.cc:1434] Started /tmp/dist-test-task_qhVVJ/build/asan/bin/kudu as pid 23845
I20260810 01:57:28.798569 22804 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-1/wal/instance
I20260810 01:57:28.806504 22804 external_mini_cluster.cc:1372] Running /tmp/dist-test-task_qhVVJ/build/asan/bin/kudu
/tmp/dist-test-task_qhVVJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.69.3:0
--local_ip_for_outbound_sockets=127.22.69.3
--webserver_interface=127.22.69.3
--webserver_port=0
--tserver_master_addrs=127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917
--builtin_ntp_servers=127.22.69.20:43209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260810 01:57:28.816246 24063 heartbeater.cc:344] Connected to a master server at 127.22.69.60:36917
I20260810 01:57:28.816736 24063 heartbeater.cc:461] Registering TS with master...
I20260810 01:57:28.818315 24063 heartbeater.cc:507] Master 127.22.69.60:36917 requested a full tablet report, sending...
I20260810 01:57:28.820334 23491 ts_manager.cc:194] Registered new tserver with Master: 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
I20260810 01:57:28.821171 23491 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.22.69.2:48785
I20260810 01:57:28.821585 24066 heartbeater.cc:344] Connected to a master server at 127.22.69.62:35311
I20260810 01:57:28.821745 24066 heartbeater.cc:461] Registering TS with master...
I20260810 01:57:28.822525 24066 heartbeater.cc:507] Master 127.22.69.62:35311 requested a full tablet report, sending...
I20260810 01:57:28.823601 23284 ts_manager.cc:194] Registered new tserver with Master: 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
I20260810 01:57:28.825999 24069 heartbeater.cc:344] Connected to a master server at 127.22.69.61:41945
I20260810 01:57:28.826170 24069 heartbeater.cc:461] Registering TS with master...
I20260810 01:57:28.826509 24069 heartbeater.cc:507] Master 127.22.69.61:41945 requested a full tablet report, sending...
I20260810 01:57:28.827739 23392 ts_manager.cc:194] Registered new tserver with Master: 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
I20260810 01:57:28.927019 24084 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 757bb3f336084510845ed58df264c721)
I20260810 01:57:28.927268 24084 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:28.927940 24084 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
I20260810 01:57:28.928689 23404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "919313c0dee540a690f46dbc8208d20b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1422349180c446db6c1768b3025253c" is_pre_election: true
I20260810 01:57:28.928673 23506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "919313c0dee540a690f46dbc8208d20b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "757bb3f336084510845ed58df264c721" is_pre_election: true
I20260810 01:57:28.929418 23260 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 01:57:28.929793 24084 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 01:57:29.046617 24086 raft_consensus.cc:493] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 757bb3f336084510845ed58df264c721)
I20260810 01:57:29.046819 24086 raft_consensus.cc:515] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:29.047571 24086 leader_election.cc:290] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
I20260810 01:57:29.048009 23302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b1422349180c446db6c1768b3025253c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "919313c0dee540a690f46dbc8208d20b" is_pre_election: true
I20260810 01:57:29.048111 23506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b1422349180c446db6c1768b3025253c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "757bb3f336084510845ed58df264c721" is_pre_election: true
I20260810 01:57:29.048848 23368 leader_election.cc:304] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1422349180c446db6c1768b3025253c; no voters: 757bb3f336084510845ed58df264c721, 919313c0dee540a690f46dbc8208d20b
I20260810 01:57:29.049196 24086 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 01:57:29.203226 24083 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:57:29.203554 24083 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:57:29.203763 24083 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260810 01:57:29.213370 24083 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:57:29.213567 24083 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.69.3
I20260810 01:57:29.224654 24083 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.69.20:43209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.69.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.69.3
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.69.3
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 10 Aug 2026 01:44:04 UTC on e7f111948823
build id 13266
ASAN enabled
I20260810 01:57:29.226496 24083 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:57:29.228291 24083 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260810 01:57:29.238278 24094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:57:29.238317 24095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:57:29.239615 24097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:57:29.240020 24083 server_base.cc:1061] running on GCE node
I20260810 01:57:29.240623 24083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:57:29.241603 24083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:57:29.242826 24083 hybrid_clock.cc:648] HybridClock initialized: now 1786327049242769 us; error 43 us; skew 500 ppm
I20260810 01:57:29.243180 24083 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 01:57:29.245432 24083 webserver.cc:533] Webserver started at http://127.22.69.3:32869/ using document root <none> and password file <none>
I20260810 01:57:29.246182 24083 fs_manager.cc:362] Metadata directory not provided
I20260810 01:57:29.246326 24083 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:57:29.246708 24083 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 01:57:29.249326 24083 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-2/data/instance:
uuid: "498d352dbea9405199a121e84a4ab926"
format_stamp: "Formatted at 2026-08-10 01:57:29 on dist-test-slave-n326"
I20260810 01:57:29.250162 24083 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-2/wal/instance:
uuid: "498d352dbea9405199a121e84a4ab926"
format_stamp: "Formatted at 2026-08-10 01:57:29 on dist-test-slave-n326"
I20260810 01:57:29.256028 24083 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260810 01:57:29.259507 24105 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:57:29.260788 24083 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 01:57:29.260962 24083 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-2/data,/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-2/wal
uuid: "498d352dbea9405199a121e84a4ab926"
format_stamp: "Formatted at 2026-08-10 01:57:29 on dist-test-slave-n326"
I20260810 01:57:29.261190 24083 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:57:29.280524 24083 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260810 01:57:29.281424 24083 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:57:29.281842 24083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:57:29.283195 24083 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260810 01:57:29.285475 24083 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 01:57:29.285606 24083 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 01:57:29.285740 24083 ts_tablet_manager.cc:616] Registered 0 tablets
I20260810 01:57:29.285905 24083 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 01:57:29.324985 24063 heartbeater.cc:499] Master 127.22.69.60:36917 was elected leader, sending a full tablet report...
I20260810 01:57:29.331534 24083 rpc_server.cc:307] RPC server started. Bound to: 127.22.69.3:38597
I20260810 01:57:29.331625 24297 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.22.69.3:38597 every 8 connection(s)
I20260810 01:57:29.333092 24083 server_base.cc:1193] Dumped server information to /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-2/data/info.pb
I20260810 01:57:29.338110 22804 external_mini_cluster.cc:1434] Started /tmp/dist-test-task_qhVVJ/build/asan/bin/kudu as pid 24083
I20260810 01:57:29.338397 22804 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-2/wal/instance
I20260810 01:57:29.357760 24300 heartbeater.cc:344] Connected to a master server at 127.22.69.62:35311
I20260810 01:57:29.358165 24300 heartbeater.cc:461] Registering TS with master...
I20260810 01:57:29.359587 24300 heartbeater.cc:507] Master 127.22.69.62:35311 requested a full tablet report, sending...
I20260810 01:57:29.361560 23284 ts_manager.cc:194] Registered new tserver with Master: 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
I20260810 01:57:29.362218 24301 heartbeater.cc:344] Connected to a master server at 127.22.69.61:41945
I20260810 01:57:29.362412 24301 heartbeater.cc:461] Registering TS with master...
I20260810 01:57:29.362848 24301 heartbeater.cc:507] Master 127.22.69.61:41945 requested a full tablet report, sending...
I20260810 01:57:29.363904 24298 heartbeater.cc:344] Connected to a master server at 127.22.69.60:36917
I20260810 01:57:29.364080 24298 heartbeater.cc:461] Registering TS with master...
I20260810 01:57:29.364085 23392 ts_manager.cc:194] Registered new tserver with Master: 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
I20260810 01:57:29.364616 24298 heartbeater.cc:507] Master 127.22.69.60:36917 requested a full tablet report, sending...
I20260810 01:57:29.365516 23491 ts_manager.cc:194] Registered new tserver with Master: 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
I20260810 01:57:29.366313 23491 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.22.69.3:44651
I20260810 01:57:29.376498 22804 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260810 01:57:29.415634 23491 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56052:
name: "default.Table_72b6f93a124042e4a090dc194134c75a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:29.417532 23487 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56052:
name: "default.Table_4fb63c687c11493ca7c0b76735f137e9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:29.418296 23485 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56052:
name: "default.Table_983bf2e604df4fdbac75c5aca292f736"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:29.418969 23484 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56052:
name: "default.Table_379ea52611ef49d591a022addb7afc0c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:29.419539 23483 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56052:
name: "default.Table_73d521a2c5504815ad6d7cc3dd0bf818"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:29.415556 23489 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56052:
name: "default.Table_22726fd1a11f409f9f5287ab16d8e551"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:29.421686 23481 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56052:
name: "default.Table_0ba2046f969346f68f1bde7cf6c9f273"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:29.422370 23480 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56052:
name: "default.Table_510ae1b5fcc74db788ffb64925cad250"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:29.423604 23479 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56052:
name: "default.Table_c8ad906be0f74210912a57adead21255"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:29.425590 23476 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56052:
name: "default.Table_57697192c9bf44969c2b200e94865a3e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 01:57:29.427618 23491 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_72b6f93a124042e4a090dc194134c75a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 01:57:29.435739 23487 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_4fb63c687c11493ca7c0b76735f137e9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 01:57:29.437012 23483 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_73d521a2c5504815ad6d7cc3dd0bf818 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 01:57:29.437713 23480 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_510ae1b5fcc74db788ffb64925cad250 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 01:57:29.440088 23484 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_379ea52611ef49d591a022addb7afc0c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 01:57:29.440914 23479 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_c8ad906be0f74210912a57adead21255 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 01:57:29.441794 23481 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_0ba2046f969346f68f1bde7cf6c9f273 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 01:57:29.442760 23485 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_983bf2e604df4fdbac75c5aca292f736 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 01:57:29.443552 23489 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_22726fd1a11f409f9f5287ab16d8e551 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 01:57:29.463868 22804 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-task_qhVVJ/build/asan/bin/kudu pid 23441
I20260810 01:57:29.465387 23539 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:57:29.554236 24389 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 757bb3f336084510845ed58df264c721)
I20260810 01:57:29.554632 24389 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:29.556100 23404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "919313c0dee540a690f46dbc8208d20b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1422349180c446db6c1768b3025253c" is_pre_election: true
I20260810 01:57:29.556363 24389 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
I20260810 01:57:29.623420 24390 raft_consensus.cc:493] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 757bb3f336084510845ed58df264c721)
I20260810 01:57:29.623631 24390 raft_consensus.cc:515] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:29.624384 24390 leader_election.cc:290] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
I20260810 01:57:29.624843 23302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b1422349180c446db6c1768b3025253c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "919313c0dee540a690f46dbc8208d20b" is_pre_election: true
I20260810 01:57:29.625057 23302 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1422349180c446db6c1768b3025253c in term 1.
I20260810 01:57:29.625667 23375 leader_election.cc:304] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b, b1422349180c446db6c1768b3025253c; no voters:
I20260810 01:57:29.626196 24390 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 01:57:29.626405 24390 raft_consensus.cc:493] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 757bb3f336084510845ed58df264c721)
I20260810 01:57:29.626557 24390 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 1 FOLLOWER]: Advancing to term 2
I20260810 01:57:29.629945 24390 raft_consensus.cc:515] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:29.637975 23302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b1422349180c446db6c1768b3025253c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "919313c0dee540a690f46dbc8208d20b"
I20260810 01:57:29.638183 23302 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 1 FOLLOWER]: Advancing to term 2
I20260810 01:57:29.641530 23302 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1422349180c446db6c1768b3025253c in term 2.
I20260810 01:57:29.642172 23375 leader_election.cc:304] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b, b1422349180c446db6c1768b3025253c; no voters:
I20260810 01:57:29.647164 24392 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 2 FOLLOWER]: Leader election won for term 2
I20260810 01:57:29.647254 24390 leader_election.cc:290] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 2 election: Requested vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
I20260810 01:57:29.667968 24392 raft_consensus.cc:697] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 2 LEADER]: Becoming Leader. State: Replica: b1422349180c446db6c1768b3025253c, State: Running, Role: LEADER
I20260810 01:57:29.668655 24392 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:29.682075 23506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "919313c0dee540a690f46dbc8208d20b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "757bb3f336084510845ed58df264c721" is_pre_election: true
I20260810 01:57:29.682925 23260 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 01:57:29.683104 24390 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1422349180c446db6c1768b3025253c. Latest consensus state: current_term: 2 leader_uuid: "b1422349180c446db6c1768b3025253c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:29.683314 24389 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 01:57:29.683401 24390 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: This master's current role is: LEADER
I20260810 01:57:29.684927 23503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b1422349180c446db6c1768b3025253c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "757bb3f336084510845ed58df264c721"
I20260810 01:57:29.685674 23505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b1422349180c446db6c1768b3025253c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "757bb3f336084510845ed58df264c721" is_pre_election: true
I20260810 01:57:29.686735 23302 raft_consensus.cc:1240] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 2 FOLLOWER]: Rejecting Update request from peer 757bb3f336084510845ed58df264c721 for earlier term 1. Current term is 2. Ops: []
I20260810 01:57:29.687258 22804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task_qhVVJ/build/asan/bin/kudu with pid 23441
I20260810 01:57:29.693832 24397 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260810 01:57:29.697111 24397 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260810 01:57:29.698295 24397 catalog_manager.cc:1295] Loaded cluster ID: dfa48c56b63b4d0cbf3d6124921e8a17
I20260810 01:57:29.698387 24397 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260810 01:57:29.700022 24397 catalog_manager.cc:1540] Loading token signing keys...
I20260810 01:57:29.701274 24397 catalog_manager.cc:6103] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c: Loaded TSK: 0
I20260810 01:57:29.702555 24397 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260810 01:57:29.703688 24397 catalog_manager.cc:1563] Loading latest processed Hive Metastore notification log event ID...
I20260810 01:57:29.704803 24397 catalog_manager.cc:5941] Last processed Hive Metastore notification event ID: -1
W20260810 01:57:29.733721 24317 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.22.69.60:36917)
W20260810 01:57:29.735247 23259 connection.cc:573] server connection from 127.0.0.1:58144 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 01:57:29.739446 24329 negotiation.cc:336] Failed RPC negotiation. Trace:
0810 01:57:29.735599 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.22.69.60:36917 (local address 127.0.0.1:56064)
0810 01:57:29.735960 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0810 01:57:29.735982 (+ 22us) client_negotiation.cc:175] Beginning negotiation
0810 01:57:29.736156 (+ 174us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0810 01:57:29.737449 (+ 1293us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":101}
I20260810 01:57:29.742444 23391 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45748:
name: "default.Table_c8ad906be0f74210912a57adead21255"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:29.742766 23390 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45748:
name: "default.Table_510ae1b5fcc74db788ffb64925cad250"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:29.743120 23387 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45748:
name: "default.Table_22726fd1a11f409f9f5287ab16d8e551"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:29.744697 23382 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45748:
name: "default.Table_983bf2e604df4fdbac75c5aca292f736"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:29.744323 23381 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45748:
name: "default.Table_379ea52611ef49d591a022addb7afc0c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:29.745112 23385 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45748:
name: "default.Table_4fb63c687c11493ca7c0b76735f137e9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:29.745440 23380 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45748:
name: "default.Table_73d521a2c5504815ad6d7cc3dd0bf818"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:29.743877 23386 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45748:
name: "default.Table_72b6f93a124042e4a090dc194134c75a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:29.754976 23392 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45748:
name: "default.Table_57697192c9bf44969c2b200e94865a3e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:29.741999 23389 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45748:
name: "default.Table_0ba2046f969346f68f1bde7cf6c9f273"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 01:57:29.758574 23391 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_c8ad906be0f74210912a57adead21255 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 01:57:29.763974 23386 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_72b6f93a124042e4a090dc194134c75a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 01:57:29.765035 23381 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_379ea52611ef49d591a022addb7afc0c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 01:57:29.766064 23380 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_73d521a2c5504815ad6d7cc3dd0bf818 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 01:57:29.767148 23382 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_983bf2e604df4fdbac75c5aca292f736 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 01:57:29.768198 23390 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_510ae1b5fcc74db788ffb64925cad250 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 01:57:29.769152 23385 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_4fb63c687c11493ca7c0b76735f137e9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 01:57:29.770762 23387 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_22726fd1a11f409f9f5287ab16d8e551 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 01:57:29.772218 23392 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_57697192c9bf44969c2b200e94865a3e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 01:57:29.777751 23389 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_0ba2046f969346f68f1bde7cf6c9f273 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 01:57:29.784073 23841 heartbeater.cc:646] Failed to heartbeat to 127.22.69.60:36917 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
W20260810 01:57:29.831687 24063 heartbeater.cc:646] Failed to heartbeat to 127.22.69.60:36917 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:57:29.868443 24298 heartbeater.cc:499] Master 127.22.69.60:36917 was elected leader, sending a full tablet report...
W20260810 01:57:29.892609 24298 heartbeater.cc:646] Failed to heartbeat to 127.22.69.60:36917 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:57:29.950619 23391 catalog_manager.cc:2200] created HMS catalog entry for table [id=4da3c920ea6a48588c9f0c91da00fb69]
I20260810 01:57:29.968977 23302 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 2 FOLLOWER]: Refusing update from remote peer b1422349180c446db6c1768b3025253c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20260810 01:57:29.970053 24390 consensus_queue.cc:1048] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [LEADER]: Connected to new peer: Peer: permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260810 01:57:29.990643 24389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1422349180c446db6c1768b3025253c. Latest consensus state: current_term: 2 leader_uuid: "b1422349180c446db6c1768b3025253c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:29.990909 24389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: This master's current role is: FOLLOWER
W20260810 01:57:29.991310 23368 consensus_peers.cc:597] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c -> Peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Couldn't send request to peer 757bb3f336084510845ed58df264c721. Status: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 01:57:30.002169 24392 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "b1422349180c446db6c1768b3025253c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:30.002415 24392 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: This master's current role is: LEADER
I20260810 01:57:30.007613 23386 catalog_manager.cc:2200] created HMS catalog entry for table [id=017756566b72415aa33842f8df2b3e91]
I20260810 01:57:30.008961 24389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "b1422349180c446db6c1768b3025253c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:30.009186 24389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:57:30.013270 23391 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45748:
table { table_name: "default.Table_c8ad906be0f74210912a57adead21255" } new_table_name: "default.Table_e2cb378ea960473c833b1d18fbe32f91" modify_external_catalogs: true
I20260810 01:57:30.044237 23386 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45748:
table { table_name: "default.Table_72b6f93a124042e4a090dc194134c75a" } new_table_name: "default.Table_b8900cd2c2574605ade34fc7ccc46f94" modify_external_catalogs: true
I20260810 01:57:30.086345 23381 catalog_manager.cc:2200] created HMS catalog entry for table [id=87ba140e9080402aa494198d6f212f05]
I20260810 01:57:30.114821 23964 tablet_service.cc:1511] Processing CreateTablet for tablet b24af70442e44ca4800206b1b5bd7715 (DEFAULT_TABLE table=default.table_c8ad906be0f74210912a57adead21255 [id=4da3c920ea6a48588c9f0c91da00fb69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:30.116497 23964 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b24af70442e44ca4800206b1b5bd7715. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:30.126596 23963 tablet_service.cc:1511] Processing CreateTablet for tablet 0b876159b5cb420391459b6263472051 (DEFAULT_TABLE table=default.table_72b6f93a124042e4a090dc194134c75a [id=017756566b72415aa33842f8df2b3e91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:30.128014 23963 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b876159b5cb420391459b6263472051. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:30.134860 23381 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45748:
table { table_name: "default.Table_379ea52611ef49d591a022addb7afc0c" } new_table_name: "default.Table_09a7f5150ded428991a5d464657e85ae" modify_external_catalogs: true
I20260810 01:57:30.137605 23962 tablet_service.cc:1511] Processing CreateTablet for tablet bf1918dc18cd4be1a9a72f65cd8b2a54 (DEFAULT_TABLE table=default.table_379ea52611ef49d591a022addb7afc0c [id=87ba140e9080402aa494198d6f212f05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:30.138145 23962 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf1918dc18cd4be1a9a72f65cd8b2a54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:30.151695 23747 tablet_service.cc:1511] Processing CreateTablet for tablet b24af70442e44ca4800206b1b5bd7715 (DEFAULT_TABLE table=default.table_c8ad906be0f74210912a57adead21255 [id=4da3c920ea6a48588c9f0c91da00fb69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:30.153452 23747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b24af70442e44ca4800206b1b5bd7715. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:30.154346 24192 tablet_service.cc:1511] Processing CreateTablet for tablet b24af70442e44ca4800206b1b5bd7715 (DEFAULT_TABLE table=default.table_c8ad906be0f74210912a57adead21255 [id=4da3c920ea6a48588c9f0c91da00fb69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:30.155264 23745 tablet_service.cc:1511] Processing CreateTablet for tablet bf1918dc18cd4be1a9a72f65cd8b2a54 (DEFAULT_TABLE table=default.table_379ea52611ef49d591a022addb7afc0c [id=87ba140e9080402aa494198d6f212f05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:30.155771 23745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf1918dc18cd4be1a9a72f65cd8b2a54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:30.155977 24187 tablet_service.cc:1511] Processing CreateTablet for tablet bf1918dc18cd4be1a9a72f65cd8b2a54 (DEFAULT_TABLE table=default.table_379ea52611ef49d591a022addb7afc0c [id=87ba140e9080402aa494198d6f212f05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:30.156960 24187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf1918dc18cd4be1a9a72f65cd8b2a54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:30.159413 24191 tablet_service.cc:1511] Processing CreateTablet for tablet 0b876159b5cb420391459b6263472051 (DEFAULT_TABLE table=default.table_72b6f93a124042e4a090dc194134c75a [id=017756566b72415aa33842f8df2b3e91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:30.160012 24191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b876159b5cb420391459b6263472051. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:30.161178 23746 tablet_service.cc:1511] Processing CreateTablet for tablet 0b876159b5cb420391459b6263472051 (DEFAULT_TABLE table=default.table_72b6f93a124042e4a090dc194134c75a [id=017756566b72415aa33842f8df2b3e91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:30.161813 23746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b876159b5cb420391459b6263472051. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:30.170148 24192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b24af70442e44ca4800206b1b5bd7715. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:30.184453 24422 tablet_bootstrap.cc:492] T 0b876159b5cb420391459b6263472051 P 498d352dbea9405199a121e84a4ab926: Bootstrap starting.
I20260810 01:57:30.185600 24421 tablet_bootstrap.cc:492] T 0b876159b5cb420391459b6263472051 P 48e33bc1e8e94d4887bb64214d0b6c36: Bootstrap starting.
I20260810 01:57:30.193243 24424 tablet_bootstrap.cc:492] T 0b876159b5cb420391459b6263472051 P 50d44445234b4931aaa28141cdaa67ab: Bootstrap starting.
I20260810 01:57:30.197449 24424 tablet_bootstrap.cc:654] T 0b876159b5cb420391459b6263472051 P 50d44445234b4931aaa28141cdaa67ab: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:30.201829 23380 catalog_manager.cc:2200] created HMS catalog entry for table [id=2f1832807601444a88d61b8396a67db9]
I20260810 01:57:30.199019 24424 log.cc:826] T 0b876159b5cb420391459b6263472051 P 50d44445234b4931aaa28141cdaa67ab: Log is configured to *not* fsync() on all Append() calls
I20260810 01:57:30.209609 23380 master_service.cc:960] ReplaceTablet: received request to replace tablet b24af70442e44ca4800206b1b5bd7715 from {username='slave'} at 127.0.0.1:45748
I20260810 01:57:30.218331 23380 catalog_manager.cc:6006] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c: Sending DeleteTablet for 3 replicas of tablet b24af70442e44ca4800206b1b5bd7715
I20260810 01:57:30.219954 23747 tablet_service.cc:1558] Processing DeleteTablet for tablet b24af70442e44ca4800206b1b5bd7715 with delete_type TABLET_DATA_DELETED (replaced by tablet 1844a63489d74f1c919e96501e4bd568) from {username='slave'} at 127.0.0.1:55738
I20260810 01:57:30.223574 24422 tablet_bootstrap.cc:654] T 0b876159b5cb420391459b6263472051 P 498d352dbea9405199a121e84a4ab926: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:30.224457 24421 tablet_bootstrap.cc:654] T 0b876159b5cb420391459b6263472051 P 48e33bc1e8e94d4887bb64214d0b6c36: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:30.226297 23746 tablet_service.cc:1511] Processing CreateTablet for tablet 09ed68395e2d44a5b51a22abe2203406 (DEFAULT_TABLE table=default.table_73d521a2c5504815ad6d7cc3dd0bf818 [id=2f1832807601444a88d61b8396a67db9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:30.226756 23746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09ed68395e2d44a5b51a22abe2203406. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:30.226950 23963 tablet_service.cc:1511] Processing CreateTablet for tablet 09ed68395e2d44a5b51a22abe2203406 (DEFAULT_TABLE table=default.table_73d521a2c5504815ad6d7cc3dd0bf818 [id=2f1832807601444a88d61b8396a67db9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:30.227406 23963 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09ed68395e2d44a5b51a22abe2203406. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:30.229260 24424 tablet_bootstrap.cc:492] T 0b876159b5cb420391459b6263472051 P 50d44445234b4931aaa28141cdaa67ab: No bootstrap required, opened a new log
W20260810 01:57:30.229223 23366 catalog_manager.cc:5030] TS 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): delete failed for tablet b24af70442e44ca4800206b1b5bd7715 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b24af70442e44ca4800206b1b5bd7715 already in progress: creating tablet
I20260810 01:57:30.229569 24424 ts_tablet_manager.cc:1403] T 0b876159b5cb420391459b6263472051 P 50d44445234b4931aaa28141cdaa67ab: Time spent bootstrapping tablet: real 0.037s user 0.005s sys 0.005s
I20260810 01:57:30.230257 24422 log.cc:826] T 0b876159b5cb420391459b6263472051 P 498d352dbea9405199a121e84a4ab926: Log is configured to *not* fsync() on all Append() calls
I20260810 01:57:30.231192 24421 log.cc:826] T 0b876159b5cb420391459b6263472051 P 48e33bc1e8e94d4887bb64214d0b6c36: Log is configured to *not* fsync() on all Append() calls
I20260810 01:57:30.233415 23380 catalog_manager.cc:6807] ReplaceTablet: tablet b24af70442e44ca4800206b1b5bd7715 deleted and replaced by tablet 1844a63489d74f1c919e96501e4bd568
I20260810 01:57:30.233867 23380 master_service.cc:960] ReplaceTablet: received request to replace tablet bf1918dc18cd4be1a9a72f65cd8b2a54 from {username='slave'} at 127.0.0.1:45748
I20260810 01:57:30.233714 23964 tablet_service.cc:1558] Processing DeleteTablet for tablet b24af70442e44ca4800206b1b5bd7715 with delete_type TABLET_DATA_DELETED (replaced by tablet 1844a63489d74f1c919e96501e4bd568) from {username='slave'} at 127.0.0.1:37482
I20260810 01:57:30.237123 24424 raft_consensus.cc:359] T 0b876159b5cb420391459b6263472051 P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.237530 24424 raft_consensus.cc:385] T 0b876159b5cb420391459b6263472051 P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:30.237670 24424 raft_consensus.cc:740] T 0b876159b5cb420391459b6263472051 P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50d44445234b4931aaa28141cdaa67ab, State: Initialized, Role: FOLLOWER
I20260810 01:57:30.238444 24424 consensus_queue.cc:260] T 0b876159b5cb420391459b6263472051 P 50d44445234b4931aaa28141cdaa67ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.240885 24191 tablet_service.cc:1558] Processing DeleteTablet for tablet b24af70442e44ca4800206b1b5bd7715 with delete_type TABLET_DATA_DELETED (replaced by tablet 1844a63489d74f1c919e96501e4bd568) from {username='slave'} at 127.0.0.1:56938
I20260810 01:57:30.258911 23747 tablet_service.cc:1511] Processing CreateTablet for tablet 1844a63489d74f1c919e96501e4bd568 (DEFAULT_TABLE table=default.table_c8ad906be0f74210912a57adead21255 [id=4da3c920ea6a48588c9f0c91da00fb69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:30.259474 23747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1844a63489d74f1c919e96501e4bd568. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 01:57:30.262730 23366 catalog_manager.cc:5030] TS 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): delete failed for tablet b24af70442e44ca4800206b1b5bd7715 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b24af70442e44ca4800206b1b5bd7715 already in progress: creating tablet
W20260810 01:57:30.264029 23375 catalog_manager.cc:5030] TS 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): delete failed for tablet b24af70442e44ca4800206b1b5bd7715 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b24af70442e44ca4800206b1b5bd7715 already in progress: creating tablet
I20260810 01:57:30.267663 24421 tablet_bootstrap.cc:492] T 0b876159b5cb420391459b6263472051 P 48e33bc1e8e94d4887bb64214d0b6c36: No bootstrap required, opened a new log
I20260810 01:57:30.268034 24421 ts_tablet_manager.cc:1403] T 0b876159b5cb420391459b6263472051 P 48e33bc1e8e94d4887bb64214d0b6c36: Time spent bootstrapping tablet: real 0.083s user 0.004s sys 0.007s
I20260810 01:57:30.274777 23964 tablet_service.cc:1511] Processing CreateTablet for tablet 1844a63489d74f1c919e96501e4bd568 (DEFAULT_TABLE table=default.table_c8ad906be0f74210912a57adead21255 [id=4da3c920ea6a48588c9f0c91da00fb69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:30.276080 23380 catalog_manager.cc:6006] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c: Sending DeleteTablet for 3 replicas of tablet bf1918dc18cd4be1a9a72f65cd8b2a54
I20260810 01:57:30.276571 23964 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1844a63489d74f1c919e96501e4bd568. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:30.276901 23745 tablet_service.cc:1558] Processing DeleteTablet for tablet bf1918dc18cd4be1a9a72f65cd8b2a54 with delete_type TABLET_DATA_DELETED (replaced by tablet 1c3db8429f9f4cf5b29d3d1a6cc0651c) from {username='slave'} at 127.0.0.1:55738
I20260810 01:57:30.277091 24191 tablet_service.cc:1511] Processing CreateTablet for tablet 1844a63489d74f1c919e96501e4bd568 (DEFAULT_TABLE table=default.table_c8ad906be0f74210912a57adead21255 [id=4da3c920ea6a48588c9f0c91da00fb69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:30.277576 24191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1844a63489d74f1c919e96501e4bd568. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 01:57:30.277621 23366 catalog_manager.cc:5030] TS 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): delete failed for tablet bf1918dc18cd4be1a9a72f65cd8b2a54 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bf1918dc18cd4be1a9a72f65cd8b2a54 already in progress: creating tablet
I20260810 01:57:30.279407 23963 tablet_service.cc:1558] Processing DeleteTablet for tablet bf1918dc18cd4be1a9a72f65cd8b2a54 with delete_type TABLET_DATA_DELETED (replaced by tablet 1c3db8429f9f4cf5b29d3d1a6cc0651c) from {username='slave'} at 127.0.0.1:37482
I20260810 01:57:30.279487 23380 catalog_manager.cc:6807] ReplaceTablet: tablet bf1918dc18cd4be1a9a72f65cd8b2a54 deleted and replaced by tablet 1c3db8429f9f4cf5b29d3d1a6cc0651c
I20260810 01:57:30.280158 23380 master_service.cc:960] ReplaceTablet: received request to replace tablet 0b876159b5cb420391459b6263472051 from {username='slave'} at 127.0.0.1:45748
W20260810 01:57:30.280232 23366 catalog_manager.cc:5030] TS 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): delete failed for tablet bf1918dc18cd4be1a9a72f65cd8b2a54 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bf1918dc18cd4be1a9a72f65cd8b2a54 already in progress: creating tablet
I20260810 01:57:30.280704 24422 tablet_bootstrap.cc:492] T 0b876159b5cb420391459b6263472051 P 498d352dbea9405199a121e84a4ab926: No bootstrap required, opened a new log
I20260810 01:57:30.290336 24422 ts_tablet_manager.cc:1403] T 0b876159b5cb420391459b6263472051 P 498d352dbea9405199a121e84a4ab926: Time spent bootstrapping tablet: real 0.097s user 0.009s sys 0.002s
I20260810 01:57:30.292346 24424 ts_tablet_manager.cc:1434] T 0b876159b5cb420391459b6263472051 P 50d44445234b4931aaa28141cdaa67ab: Time spent starting tablet: real 0.063s user 0.035s sys 0.019s
I20260810 01:57:30.296021 23846 heartbeater.cc:499] Master 127.22.69.61:41945 was elected leader, sending a full tablet report...
I20260810 01:57:30.296990 24424 tablet_bootstrap.cc:492] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 50d44445234b4931aaa28141cdaa67ab: Bootstrap starting.
I20260810 01:57:30.298713 23964 tablet_service.cc:1511] Processing CreateTablet for tablet 1c3db8429f9f4cf5b29d3d1a6cc0651c (DEFAULT_TABLE table=default.table_379ea52611ef49d591a022addb7afc0c [id=87ba140e9080402aa494198d6f212f05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:30.298815 24424 tablet_bootstrap.cc:654] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 50d44445234b4931aaa28141cdaa67ab: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:30.299376 23964 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c3db8429f9f4cf5b29d3d1a6cc0651c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:30.299355 24186 tablet_service.cc:1511] Processing CreateTablet for tablet 1c3db8429f9f4cf5b29d3d1a6cc0651c (DEFAULT_TABLE table=default.table_379ea52611ef49d591a022addb7afc0c [id=87ba140e9080402aa494198d6f212f05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:30.299662 23747 tablet_service.cc:1511] Processing CreateTablet for tablet 1c3db8429f9f4cf5b29d3d1a6cc0651c (DEFAULT_TABLE table=default.table_379ea52611ef49d591a022addb7afc0c [id=87ba140e9080402aa494198d6f212f05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:30.299885 24186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c3db8429f9f4cf5b29d3d1a6cc0651c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:30.300072 23747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c3db8429f9f4cf5b29d3d1a6cc0651c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:30.301937 24421 raft_consensus.cc:359] T 0b876159b5cb420391459b6263472051 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.306684 23380 catalog_manager.cc:6006] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c: Sending DeleteTablet for 3 replicas of tablet 0b876159b5cb420391459b6263472051
I20260810 01:57:30.307403 23746 tablet_service.cc:1558] Processing DeleteTablet for tablet 0b876159b5cb420391459b6263472051 with delete_type TABLET_DATA_DELETED (replaced by tablet 16db112abfb34d4a9f2ecbee98c39d8b) from {username='slave'} at 127.0.0.1:55738
I20260810 01:57:30.307699 24426 tablet_replica.cc:333] T 0b876159b5cb420391459b6263472051 P 50d44445234b4931aaa28141cdaa67ab: stopping tablet replica
I20260810 01:57:30.308202 24426 raft_consensus.cc:2243] T 0b876159b5cb420391459b6263472051 P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 01:57:30.308451 24186 tablet_service.cc:1558] Processing DeleteTablet for tablet 0b876159b5cb420391459b6263472051 with delete_type TABLET_DATA_DELETED (replaced by tablet 16db112abfb34d4a9f2ecbee98c39d8b) from {username='slave'} at 127.0.0.1:56938
I20260810 01:57:30.308627 24426 raft_consensus.cc:2272] T 0b876159b5cb420391459b6263472051 P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 01:57:30.308671 23380 catalog_manager.cc:6807] ReplaceTablet: tablet 0b876159b5cb420391459b6263472051 deleted and replaced by tablet 16db112abfb34d4a9f2ecbee98c39d8b
W20260810 01:57:30.309317 23375 catalog_manager.cc:5030] TS 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): delete failed for tablet 0b876159b5cb420391459b6263472051 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0b876159b5cb420391459b6263472051 already in progress: creating tablet
I20260810 01:57:30.310045 24422 raft_consensus.cc:359] T 0b876159b5cb420391459b6263472051 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.310432 24426 ts_tablet_manager.cc:1916] T 0b876159b5cb420391459b6263472051 P 50d44445234b4931aaa28141cdaa67ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:57:30.311244 23380 master_service.cc:960] ReplaceTablet: received request to replace tablet b24af70442e44ca4800206b1b5bd7715 from {username='slave'} at 127.0.0.1:45748
I20260810 01:57:30.308138 23963 tablet_service.cc:1558] Processing DeleteTablet for tablet 0b876159b5cb420391459b6263472051 with delete_type TABLET_DATA_DELETED (replaced by tablet 16db112abfb34d4a9f2ecbee98c39d8b) from {username='slave'} at 127.0.0.1:37482
I20260810 01:57:30.312028 24421 raft_consensus.cc:385] T 0b876159b5cb420391459b6263472051 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:30.312287 24421 raft_consensus.cc:740] T 0b876159b5cb420391459b6263472051 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48e33bc1e8e94d4887bb64214d0b6c36, State: Initialized, Role: FOLLOWER
W20260810 01:57:30.312288 23366 catalog_manager.cc:5030] TS 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): delete failed for tablet 0b876159b5cb420391459b6263472051 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0b876159b5cb420391459b6263472051 already in progress: creating tablet
I20260810 01:57:30.312804 24187 tablet_service.cc:1511] Processing CreateTablet for tablet 09ed68395e2d44a5b51a22abe2203406 (DEFAULT_TABLE table=default.table_73d521a2c5504815ad6d7cc3dd0bf818 [id=2f1832807601444a88d61b8396a67db9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:30.312809 24422 raft_consensus.cc:385] T 0b876159b5cb420391459b6263472051 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:30.313527 24421 consensus_queue.cc:260] T 0b876159b5cb420391459b6263472051 P 48e33bc1e8e94d4887bb64214d0b6c36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.313830 24422 raft_consensus.cc:740] T 0b876159b5cb420391459b6263472051 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 498d352dbea9405199a121e84a4ab926, State: Initialized, Role: FOLLOWER
I20260810 01:57:30.315716 24422 consensus_queue.cc:260] T 0b876159b5cb420391459b6263472051 P 498d352dbea9405199a121e84a4ab926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.317440 24426 ts_tablet_manager.cc:1929] T 0b876159b5cb420391459b6263472051 P 50d44445234b4931aaa28141cdaa67ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 01:57:30.313197 24187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09ed68395e2d44a5b51a22abe2203406. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:30.317596 24426 log.cc:1199] T 0b876159b5cb420391459b6263472051 P 50d44445234b4931aaa28141cdaa67ab: Deleting WAL directory at /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-0/wal/wals/0b876159b5cb420391459b6263472051
I20260810 01:57:30.318027 24426 ts_tablet_manager.cc:1950] T 0b876159b5cb420391459b6263472051 P 50d44445234b4931aaa28141cdaa67ab: Deleting consensus metadata
I20260810 01:57:30.319710 23366 catalog_manager.cc:5050] TS 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): tablet 0b876159b5cb420391459b6263472051 (table default.table_72b6f93a124042e4a090dc194134c75a [id=017756566b72415aa33842f8df2b3e91]) successfully deleted
I20260810 01:57:30.321313 23746 tablet_service.cc:1511] Processing CreateTablet for tablet 16db112abfb34d4a9f2ecbee98c39d8b (DEFAULT_TABLE table=default.table_72b6f93a124042e4a090dc194134c75a [id=017756566b72415aa33842f8df2b3e91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:30.321781 23746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16db112abfb34d4a9f2ecbee98c39d8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:30.325158 24422 ts_tablet_manager.cc:1434] T 0b876159b5cb420391459b6263472051 P 498d352dbea9405199a121e84a4ab926: Time spent starting tablet: real 0.035s user 0.007s sys 0.003s
I20260810 01:57:30.325431 24301 heartbeater.cc:499] Master 127.22.69.61:41945 was elected leader, sending a full tablet report...
I20260810 01:57:30.325158 24192 tablet_service.cc:1558] Processing DeleteTablet for tablet bf1918dc18cd4be1a9a72f65cd8b2a54 with delete_type TABLET_DATA_DELETED (replaced by tablet 1c3db8429f9f4cf5b29d3d1a6cc0651c) from {username='slave'} at 127.0.0.1:56938
I20260810 01:57:30.326248 23380 master_service.cc:960] ReplaceTablet: received request to replace tablet bf1918dc18cd4be1a9a72f65cd8b2a54 from {username='slave'} at 127.0.0.1:45748
I20260810 01:57:30.326185 23963 tablet_service.cc:1511] Processing CreateTablet for tablet 16db112abfb34d4a9f2ecbee98c39d8b (DEFAULT_TABLE table=default.table_72b6f93a124042e4a090dc194134c75a [id=017756566b72415aa33842f8df2b3e91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:30.326570 23963 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16db112abfb34d4a9f2ecbee98c39d8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:30.328907 23380 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45748:
name: "default.Table_73822ba0bac443d19cc7c66f6f547bda"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 01:57:30.331317 23375 catalog_manager.cc:5030] TS 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): delete failed for tablet bf1918dc18cd4be1a9a72f65cd8b2a54 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bf1918dc18cd4be1a9a72f65cd8b2a54 already in progress: creating tablet
I20260810 01:57:30.332127 24192 tablet_service.cc:1511] Processing CreateTablet for tablet 16db112abfb34d4a9f2ecbee98c39d8b (DEFAULT_TABLE table=default.table_72b6f93a124042e4a090dc194134c75a [id=017756566b72415aa33842f8df2b3e91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:30.332535 24192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16db112abfb34d4a9f2ecbee98c39d8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:30.334565 24422 tablet_bootstrap.cc:492] T b24af70442e44ca4800206b1b5bd7715 P 498d352dbea9405199a121e84a4ab926: Bootstrap starting.
I20260810 01:57:30.340222 24422 tablet_bootstrap.cc:654] T b24af70442e44ca4800206b1b5bd7715 P 498d352dbea9405199a121e84a4ab926: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:30.341517 24421 ts_tablet_manager.cc:1434] T 0b876159b5cb420391459b6263472051 P 48e33bc1e8e94d4887bb64214d0b6c36: Time spent starting tablet: real 0.073s user 0.006s sys 0.003s
I20260810 01:57:30.341852 24069 heartbeater.cc:499] Master 127.22.69.61:41945 was elected leader, sending a full tablet report...
I20260810 01:57:30.343580 24421 tablet_bootstrap.cc:492] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 48e33bc1e8e94d4887bb64214d0b6c36: Bootstrap starting.
I20260810 01:57:30.346063 24421 tablet_bootstrap.cc:654] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 48e33bc1e8e94d4887bb64214d0b6c36: Neither blocks nor log segments found. Creating new log.
W20260810 01:57:30.342815 24302 tablet.cc:2406] T 0b876159b5cb420391459b6263472051 P 498d352dbea9405199a121e84a4ab926: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 01:57:30.354830 24424 tablet_bootstrap.cc:492] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 50d44445234b4931aaa28141cdaa67ab: No bootstrap required, opened a new log
I20260810 01:57:30.355024 24424 ts_tablet_manager.cc:1403] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 50d44445234b4931aaa28141cdaa67ab: Time spent bootstrapping tablet: real 0.058s user 0.001s sys 0.003s
I20260810 01:57:30.359511 24421 tablet_bootstrap.cc:492] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 48e33bc1e8e94d4887bb64214d0b6c36: No bootstrap required, opened a new log
I20260810 01:57:30.359709 24421 ts_tablet_manager.cc:1403] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 48e33bc1e8e94d4887bb64214d0b6c36: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.000s
I20260810 01:57:30.360498 24421 raft_consensus.cc:359] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.360687 24421 raft_consensus.cc:385] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:30.360777 24421 raft_consensus.cc:740] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48e33bc1e8e94d4887bb64214d0b6c36, State: Initialized, Role: FOLLOWER
I20260810 01:57:30.361061 24421 consensus_queue.cc:260] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 48e33bc1e8e94d4887bb64214d0b6c36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.362114 24422 tablet_bootstrap.cc:492] T b24af70442e44ca4800206b1b5bd7715 P 498d352dbea9405199a121e84a4ab926: No bootstrap required, opened a new log
I20260810 01:57:30.362310 24422 ts_tablet_manager.cc:1403] T b24af70442e44ca4800206b1b5bd7715 P 498d352dbea9405199a121e84a4ab926: Time spent bootstrapping tablet: real 0.037s user 0.003s sys 0.003s
I20260810 01:57:30.363346 24422 raft_consensus.cc:359] T b24af70442e44ca4800206b1b5bd7715 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:30.363613 24422 raft_consensus.cc:385] T b24af70442e44ca4800206b1b5bd7715 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:30.363759 24422 raft_consensus.cc:740] T b24af70442e44ca4800206b1b5bd7715 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 498d352dbea9405199a121e84a4ab926, State: Initialized, Role: FOLLOWER
I20260810 01:57:30.364063 24421 ts_tablet_manager.cc:1434] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 48e33bc1e8e94d4887bb64214d0b6c36: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260810 01:57:30.364174 24422 consensus_queue.cc:260] T b24af70442e44ca4800206b1b5bd7715 P 498d352dbea9405199a121e84a4ab926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:30.364789 24422 ts_tablet_manager.cc:1434] T b24af70442e44ca4800206b1b5bd7715 P 498d352dbea9405199a121e84a4ab926: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 01:57:30.365077 23382 catalog_manager.cc:2200] created HMS catalog entry for table [id=552771dde220474a9be9abeefd3a0472]
I20260810 01:57:30.365703 24422 tablet_bootstrap.cc:492] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 498d352dbea9405199a121e84a4ab926: Bootstrap starting.
I20260810 01:57:30.366510 24421 tablet_bootstrap.cc:492] T b24af70442e44ca4800206b1b5bd7715 P 48e33bc1e8e94d4887bb64214d0b6c36: Bootstrap starting.
I20260810 01:57:30.369561 24422 tablet_bootstrap.cc:654] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 498d352dbea9405199a121e84a4ab926: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:30.373591 24421 tablet_bootstrap.cc:654] T b24af70442e44ca4800206b1b5bd7715 P 48e33bc1e8e94d4887bb64214d0b6c36: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:30.369625 24424 raft_consensus.cc:359] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.374136 24424 raft_consensus.cc:385] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:30.374265 24424 raft_consensus.cc:740] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50d44445234b4931aaa28141cdaa67ab, State: Initialized, Role: FOLLOWER
I20260810 01:57:30.374534 24424 consensus_queue.cc:260] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 50d44445234b4931aaa28141cdaa67ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.375269 24424 ts_tablet_manager.cc:1434] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 50d44445234b4931aaa28141cdaa67ab: Time spent starting tablet: real 0.020s user 0.002s sys 0.000s
I20260810 01:57:30.375531 24424 tablet_bootstrap.cc:492] T b24af70442e44ca4800206b1b5bd7715 P 50d44445234b4931aaa28141cdaa67ab: Bootstrap starting.
I20260810 01:57:30.377238 24424 tablet_bootstrap.cc:654] T b24af70442e44ca4800206b1b5bd7715 P 50d44445234b4931aaa28141cdaa67ab: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:30.382838 24422 tablet_bootstrap.cc:492] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 498d352dbea9405199a121e84a4ab926: No bootstrap required, opened a new log
I20260810 01:57:30.383150 24422 ts_tablet_manager.cc:1403] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 498d352dbea9405199a121e84a4ab926: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.004s
I20260810 01:57:30.384121 24422 raft_consensus.cc:359] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.384444 24422 raft_consensus.cc:385] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:30.384586 24422 raft_consensus.cc:740] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 498d352dbea9405199a121e84a4ab926, State: Initialized, Role: FOLLOWER
I20260810 01:57:30.386958 23746 tablet_service.cc:1511] Processing CreateTablet for tablet e6ef3afe40a847dbb08792578d52401d (DEFAULT_TABLE table=default.table_983bf2e604df4fdbac75c5aca292f736 [id=552771dde220474a9be9abeefd3a0472]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:30.386957 23964 tablet_service.cc:1511] Processing CreateTablet for tablet e6ef3afe40a847dbb08792578d52401d (DEFAULT_TABLE table=default.table_983bf2e604df4fdbac75c5aca292f736 [id=552771dde220474a9be9abeefd3a0472]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:30.387437 23964 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6ef3afe40a847dbb08792578d52401d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:30.387486 23746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6ef3afe40a847dbb08792578d52401d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:30.404646 24424 tablet_bootstrap.cc:492] T b24af70442e44ca4800206b1b5bd7715 P 50d44445234b4931aaa28141cdaa67ab: No bootstrap required, opened a new log
I20260810 01:57:30.404883 24424 ts_tablet_manager.cc:1403] T b24af70442e44ca4800206b1b5bd7715 P 50d44445234b4931aaa28141cdaa67ab: Time spent bootstrapping tablet: real 0.029s user 0.004s sys 0.011s
I20260810 01:57:30.405997 24424 raft_consensus.cc:359] T b24af70442e44ca4800206b1b5bd7715 P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:30.406213 24424 raft_consensus.cc:385] T b24af70442e44ca4800206b1b5bd7715 P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:30.406291 24424 raft_consensus.cc:740] T b24af70442e44ca4800206b1b5bd7715 P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50d44445234b4931aaa28141cdaa67ab, State: Initialized, Role: FOLLOWER
I20260810 01:57:30.406400 23964 tablet_service.cc:1558] Processing DeleteTablet for tablet bf1918dc18cd4be1a9a72f65cd8b2a54 with delete_type TABLET_DATA_DELETED (replaced by tablet 1c3db8429f9f4cf5b29d3d1a6cc0651c) from {username='slave'} at 127.0.0.1:37482
I20260810 01:57:30.392200 24187 tablet_service.cc:1511] Processing CreateTablet for tablet e6ef3afe40a847dbb08792578d52401d (DEFAULT_TABLE table=default.table_983bf2e604df4fdbac75c5aca292f736 [id=552771dde220474a9be9abeefd3a0472]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:30.406720 24429 tablet_replica.cc:333] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 48e33bc1e8e94d4887bb64214d0b6c36: stopping tablet replica
I20260810 01:57:30.406989 24187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6ef3afe40a847dbb08792578d52401d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:30.407249 24429 raft_consensus.cc:2243] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 01:57:30.407667 24429 raft_consensus.cc:2272] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 01:57:30.409461 24429 ts_tablet_manager.cc:1916] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 48e33bc1e8e94d4887bb64214d0b6c36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:57:30.396044 24192 tablet_service.cc:1558] Processing DeleteTablet for tablet b24af70442e44ca4800206b1b5bd7715 with delete_type TABLET_DATA_DELETED (replaced by tablet 1844a63489d74f1c919e96501e4bd568) from {username='slave'} at 127.0.0.1:56938
I20260810 01:57:30.396234 24191 tablet_service.cc:1558] Processing DeleteTablet for tablet 0b876159b5cb420391459b6263472051 with delete_type TABLET_DATA_DELETED (replaced by tablet 16db112abfb34d4a9f2ecbee98c39d8b) from {username='slave'} at 127.0.0.1:56938
I20260810 01:57:30.396389 24186 tablet_service.cc:1558] Processing DeleteTablet for tablet bf1918dc18cd4be1a9a72f65cd8b2a54 with delete_type TABLET_DATA_DELETED (replaced by tablet 1c3db8429f9f4cf5b29d3d1a6cc0651c) from {username='slave'} at 127.0.0.1:56938
I20260810 01:57:30.410734 24431 tablet_replica.cc:333] T b24af70442e44ca4800206b1b5bd7715 P 498d352dbea9405199a121e84a4ab926: stopping tablet replica
I20260810 01:57:30.411499 24431 raft_consensus.cc:2243] T b24af70442e44ca4800206b1b5bd7715 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 01:57:30.411878 24431 raft_consensus.cc:2272] T b24af70442e44ca4800206b1b5bd7715 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 01:57:30.413525 24431 ts_tablet_manager.cc:1916] T b24af70442e44ca4800206b1b5bd7715 P 498d352dbea9405199a121e84a4ab926: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:57:30.417551 24422 consensus_queue.cc:260] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 498d352dbea9405199a121e84a4ab926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.418258 24422 ts_tablet_manager.cc:1434] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 498d352dbea9405199a121e84a4ab926: Time spent starting tablet: real 0.035s user 0.002s sys 0.000s
I20260810 01:57:30.419129 23964 tablet_service.cc:1558] Processing DeleteTablet for tablet 0b876159b5cb420391459b6263472051 with delete_type TABLET_DATA_DELETED (replaced by tablet 16db112abfb34d4a9f2ecbee98c39d8b) from {username='slave'} at 127.0.0.1:37482
I20260810 01:57:30.419931 23964 tablet_service.cc:1558] Processing DeleteTablet for tablet b24af70442e44ca4800206b1b5bd7715 with delete_type TABLET_DATA_DELETED (replaced by tablet 1844a63489d74f1c919e96501e4bd568) from {username='slave'} at 127.0.0.1:37482
I20260810 01:57:30.420418 24422 tablet_bootstrap.cc:492] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 498d352dbea9405199a121e84a4ab926: Bootstrap starting.
I20260810 01:57:30.423925 24422 tablet_bootstrap.cc:654] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 498d352dbea9405199a121e84a4ab926: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:30.425022 24421 tablet_bootstrap.cc:492] T b24af70442e44ca4800206b1b5bd7715 P 48e33bc1e8e94d4887bb64214d0b6c36: No bootstrap required, opened a new log
I20260810 01:57:30.425177 24429 ts_tablet_manager.cc:1929] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 48e33bc1e8e94d4887bb64214d0b6c36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 01:57:30.425381 24429 log.cc:1199] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 48e33bc1e8e94d4887bb64214d0b6c36: Deleting WAL directory at /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-1/wal/wals/bf1918dc18cd4be1a9a72f65cd8b2a54
I20260810 01:57:30.425879 24429 ts_tablet_manager.cc:1950] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 48e33bc1e8e94d4887bb64214d0b6c36: Deleting consensus metadata
I20260810 01:57:30.427053 23746 tablet_service.cc:1558] Processing DeleteTablet for tablet bf1918dc18cd4be1a9a72f65cd8b2a54 with delete_type TABLET_DATA_DELETED (replaced by tablet 1c3db8429f9f4cf5b29d3d1a6cc0651c) from {username='slave'} at 127.0.0.1:55738
I20260810 01:57:30.427700 24426 tablet_replica.cc:333] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 50d44445234b4931aaa28141cdaa67ab: stopping tablet replica
I20260810 01:57:30.427740 24431 ts_tablet_manager.cc:1929] T b24af70442e44ca4800206b1b5bd7715 P 498d352dbea9405199a121e84a4ab926: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 01:57:30.427887 24431 log.cc:1199] T b24af70442e44ca4800206b1b5bd7715 P 498d352dbea9405199a121e84a4ab926: Deleting WAL directory at /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-2/wal/wals/b24af70442e44ca4800206b1b5bd7715
I20260810 01:57:30.427918 24426 raft_consensus.cc:2243] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 01:57:30.428110 24426 raft_consensus.cc:2272] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 01:57:30.428346 24431 ts_tablet_manager.cc:1950] T b24af70442e44ca4800206b1b5bd7715 P 498d352dbea9405199a121e84a4ab926: Deleting consensus metadata
I20260810 01:57:30.429041 24426 ts_tablet_manager.cc:1916] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 50d44445234b4931aaa28141cdaa67ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:57:30.430037 24431 tablet_replica.cc:333] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 498d352dbea9405199a121e84a4ab926: stopping tablet replica
I20260810 01:57:30.430212 24431 raft_consensus.cc:2243] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 01:57:30.430359 24431 raft_consensus.cc:2272] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 01:57:30.430567 24424 consensus_queue.cc:260] T b24af70442e44ca4800206b1b5bd7715 P 50d44445234b4931aaa28141cdaa67ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:30.425221 24421 ts_tablet_manager.cc:1403] T b24af70442e44ca4800206b1b5bd7715 P 48e33bc1e8e94d4887bb64214d0b6c36: Time spent bootstrapping tablet: real 0.059s user 0.010s sys 0.018s
I20260810 01:57:30.431488 23375 catalog_manager.cc:5050] TS 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): tablet b24af70442e44ca4800206b1b5bd7715 (table default.table_c8ad906be0f74210912a57adead21255 [id=4da3c920ea6a48588c9f0c91da00fb69]) successfully deleted
I20260810 01:57:30.431677 24421 raft_consensus.cc:359] T b24af70442e44ca4800206b1b5bd7715 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:30.431903 24421 raft_consensus.cc:385] T b24af70442e44ca4800206b1b5bd7715 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:30.432005 24421 raft_consensus.cc:740] T b24af70442e44ca4800206b1b5bd7715 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48e33bc1e8e94d4887bb64214d0b6c36, State: Initialized, Role: FOLLOWER
I20260810 01:57:30.436158 24424 ts_tablet_manager.cc:1434] T b24af70442e44ca4800206b1b5bd7715 P 50d44445234b4931aaa28141cdaa67ab: Time spent starting tablet: real 0.031s user 0.002s sys 0.001s
I20260810 01:57:30.438711 24426 ts_tablet_manager.cc:1929] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 50d44445234b4931aaa28141cdaa67ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 01:57:30.438956 24426 log.cc:1199] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 50d44445234b4931aaa28141cdaa67ab: Deleting WAL directory at /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-0/wal/wals/bf1918dc18cd4be1a9a72f65cd8b2a54
I20260810 01:57:30.439476 24426 ts_tablet_manager.cc:1950] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 50d44445234b4931aaa28141cdaa67ab: Deleting consensus metadata
I20260810 01:57:30.440361 24424 tablet_bootstrap.cc:492] T 09ed68395e2d44a5b51a22abe2203406 P 50d44445234b4931aaa28141cdaa67ab: Bootstrap starting.
I20260810 01:57:30.440651 23366 catalog_manager.cc:5050] TS 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): tablet bf1918dc18cd4be1a9a72f65cd8b2a54 (table default.table_379ea52611ef49d591a022addb7afc0c [id=87ba140e9080402aa494198d6f212f05]) successfully deleted
I20260810 01:57:30.442055 24424 tablet_bootstrap.cc:654] T 09ed68395e2d44a5b51a22abe2203406 P 50d44445234b4931aaa28141cdaa67ab: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:30.443567 23747 tablet_service.cc:1558] Processing DeleteTablet for tablet b24af70442e44ca4800206b1b5bd7715 with delete_type TABLET_DATA_DELETED (replaced by tablet 1844a63489d74f1c919e96501e4bd568) from {username='slave'} at 127.0.0.1:55738
I20260810 01:57:30.443898 23746 tablet_service.cc:1558] Processing DeleteTablet for tablet 0b876159b5cb420391459b6263472051 with delete_type TABLET_DATA_DELETED (replaced by tablet 16db112abfb34d4a9f2ecbee98c39d8b) from {username='slave'} at 127.0.0.1:55738
I20260810 01:57:30.444185 24426 tablet_replica.cc:333] T b24af70442e44ca4800206b1b5bd7715 P 50d44445234b4931aaa28141cdaa67ab: stopping tablet replica
I20260810 01:57:30.444420 24426 raft_consensus.cc:2243] T b24af70442e44ca4800206b1b5bd7715 P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 01:57:30.444672 24426 raft_consensus.cc:2272] T b24af70442e44ca4800206b1b5bd7715 P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 01:57:30.445624 24426 ts_tablet_manager.cc:1916] T b24af70442e44ca4800206b1b5bd7715 P 50d44445234b4931aaa28141cdaa67ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:57:30.445184 24187 tablet_service.cc:1558] Processing DeleteTablet for tablet b24af70442e44ca4800206b1b5bd7715 with delete_type TABLET_DATA_DELETED (replaced by tablet 1844a63489d74f1c919e96501e4bd568) from {username='slave'} at 127.0.0.1:56938
I20260810 01:57:30.447176 24421 consensus_queue.cc:260] T b24af70442e44ca4800206b1b5bd7715 P 48e33bc1e8e94d4887bb64214d0b6c36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:30.447683 24431 ts_tablet_manager.cc:1916] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 498d352dbea9405199a121e84a4ab926: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:57:30.447809 24421 ts_tablet_manager.cc:1434] T b24af70442e44ca4800206b1b5bd7715 P 48e33bc1e8e94d4887bb64214d0b6c36: Time spent starting tablet: real 0.017s user 0.002s sys 0.000s
I20260810 01:57:30.448117 24421 tablet_bootstrap.cc:492] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36: Bootstrap starting.
I20260810 01:57:30.452277 24421 tablet_bootstrap.cc:654] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:30.452380 24426 ts_tablet_manager.cc:1929] T b24af70442e44ca4800206b1b5bd7715 P 50d44445234b4931aaa28141cdaa67ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 01:57:30.452519 24426 log.cc:1199] T b24af70442e44ca4800206b1b5bd7715 P 50d44445234b4931aaa28141cdaa67ab: Deleting WAL directory at /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-0/wal/wals/b24af70442e44ca4800206b1b5bd7715
I20260810 01:57:30.452795 24422 tablet_bootstrap.cc:492] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 498d352dbea9405199a121e84a4ab926: No bootstrap required, opened a new log
I20260810 01:57:30.452711 23382 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45748:
table { table_name: "default.Table_73d521a2c5504815ad6d7cc3dd0bf818" } new_table_name: "default.Table_f5851b3fa264482388e4e50f34d4e37c" modify_external_catalogs: true
I20260810 01:57:30.452922 24426 ts_tablet_manager.cc:1950] T b24af70442e44ca4800206b1b5bd7715 P 50d44445234b4931aaa28141cdaa67ab: Deleting consensus metadata
I20260810 01:57:30.453495 24422 ts_tablet_manager.cc:1403] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 498d352dbea9405199a121e84a4ab926: Time spent bootstrapping tablet: real 0.033s user 0.000s sys 0.003s
I20260810 01:57:30.453938 23366 catalog_manager.cc:5050] TS 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): tablet b24af70442e44ca4800206b1b5bd7715 (table default.table_c8ad906be0f74210912a57adead21255 [id=4da3c920ea6a48588c9f0c91da00fb69]) successfully deleted
I20260810 01:57:30.454706 24422 raft_consensus.cc:359] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.454960 24422 raft_consensus.cc:385] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260810 01:57:30.454962 23366 catalog_manager.cc:5017] TS 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): delete failed for tablet 0b876159b5cb420391459b6263472051 because the tablet was not found. No further retry: Not found: Tablet not found: 0b876159b5cb420391459b6263472051
I20260810 01:57:30.455148 24424 tablet_bootstrap.cc:492] T 09ed68395e2d44a5b51a22abe2203406 P 50d44445234b4931aaa28141cdaa67ab: No bootstrap required, opened a new log
I20260810 01:57:30.455283 24424 ts_tablet_manager.cc:1403] T 09ed68395e2d44a5b51a22abe2203406 P 50d44445234b4931aaa28141cdaa67ab: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.001s
I20260810 01:57:30.456108 24424 raft_consensus.cc:359] T 09ed68395e2d44a5b51a22abe2203406 P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.456293 24424 raft_consensus.cc:385] T 09ed68395e2d44a5b51a22abe2203406 P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:30.456383 24424 raft_consensus.cc:740] T 09ed68395e2d44a5b51a22abe2203406 P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50d44445234b4931aaa28141cdaa67ab, State: Initialized, Role: FOLLOWER
I20260810 01:57:30.456377 23390 catalog_manager.cc:2200] created HMS catalog entry for table [id=7d240d10089c47eaa014d59c4ac7f0b4]
I20260810 01:57:30.456656 24424 consensus_queue.cc:260] T 09ed68395e2d44a5b51a22abe2203406 P 50d44445234b4931aaa28141cdaa67ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.460548 24422 raft_consensus.cc:740] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 498d352dbea9405199a121e84a4ab926, State: Initialized, Role: FOLLOWER
I20260810 01:57:30.460680 24424 ts_tablet_manager.cc:1434] T 09ed68395e2d44a5b51a22abe2203406 P 50d44445234b4931aaa28141cdaa67ab: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260810 01:57:30.460964 24422 consensus_queue.cc:260] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 498d352dbea9405199a121e84a4ab926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.461623 24431 ts_tablet_manager.cc:1929] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 498d352dbea9405199a121e84a4ab926: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 01:57:30.461778 24431 log.cc:1199] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 498d352dbea9405199a121e84a4ab926: Deleting WAL directory at /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-2/wal/wals/bf1918dc18cd4be1a9a72f65cd8b2a54
I20260810 01:57:30.462208 24431 ts_tablet_manager.cc:1950] T bf1918dc18cd4be1a9a72f65cd8b2a54 P 498d352dbea9405199a121e84a4ab926: Deleting consensus metadata
I20260810 01:57:30.462947 24422 ts_tablet_manager.cc:1434] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 498d352dbea9405199a121e84a4ab926: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20260810 01:57:30.463567 23375 catalog_manager.cc:5050] TS 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): tablet bf1918dc18cd4be1a9a72f65cd8b2a54 (table default.table_379ea52611ef49d591a022addb7afc0c [id=87ba140e9080402aa494198d6f212f05]) successfully deleted
I20260810 01:57:30.464021 23390 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45748:
name: "default.Table_70288f170151490fb6ba63a3bf29f01e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:30.464553 24422 tablet_bootstrap.cc:492] T 1844a63489d74f1c919e96501e4bd568 P 498d352dbea9405199a121e84a4ab926: Bootstrap starting.
I20260810 01:57:30.463236 24431 tablet_replica.cc:333] T 0b876159b5cb420391459b6263472051 P 498d352dbea9405199a121e84a4ab926: stopping tablet replica
I20260810 01:57:30.469780 24431 raft_consensus.cc:2243] T 0b876159b5cb420391459b6263472051 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 01:57:30.470001 24431 raft_consensus.cc:2272] T 0b876159b5cb420391459b6263472051 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 01:57:30.470134 23366 catalog_manager.cc:5050] TS 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): tablet bf1918dc18cd4be1a9a72f65cd8b2a54 (table default.table_379ea52611ef49d591a022addb7afc0c [id=87ba140e9080402aa494198d6f212f05]) successfully deleted
I20260810 01:57:30.471365 24422 tablet_bootstrap.cc:654] T 1844a63489d74f1c919e96501e4bd568 P 498d352dbea9405199a121e84a4ab926: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:30.472410 24429 tablet_replica.cc:333] T 0b876159b5cb420391459b6263472051 P 48e33bc1e8e94d4887bb64214d0b6c36: stopping tablet replica
I20260810 01:57:30.472476 24431 ts_tablet_manager.cc:1916] T 0b876159b5cb420391459b6263472051 P 498d352dbea9405199a121e84a4ab926: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:57:30.472623 24429 raft_consensus.cc:2243] T 0b876159b5cb420391459b6263472051 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 01:57:30.472781 24429 raft_consensus.cc:2272] T 0b876159b5cb420391459b6263472051 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 01:57:30.473531 24429 ts_tablet_manager.cc:1916] T 0b876159b5cb420391459b6263472051 P 48e33bc1e8e94d4887bb64214d0b6c36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:57:30.474319 24424 tablet_bootstrap.cc:492] T 1844a63489d74f1c919e96501e4bd568 P 50d44445234b4931aaa28141cdaa67ab: Bootstrap starting.
I20260810 01:57:30.474793 23746 tablet_service.cc:1511] Processing CreateTablet for tablet 89d9556cc526444299c786de606bb5b1 (DEFAULT_TABLE table=default.table_510ae1b5fcc74db788ffb64925cad250 [id=7d240d10089c47eaa014d59c4ac7f0b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:30.475284 23746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89d9556cc526444299c786de606bb5b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:30.476210 24424 tablet_bootstrap.cc:654] T 1844a63489d74f1c919e96501e4bd568 P 50d44445234b4931aaa28141cdaa67ab: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:30.478128 24421 tablet_bootstrap.cc:492] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36: No bootstrap required, opened a new log
I20260810 01:57:30.478340 24421 ts_tablet_manager.cc:1403] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36: Time spent bootstrapping tablet: real 0.030s user 0.004s sys 0.000s
I20260810 01:57:30.479317 24421 raft_consensus.cc:359] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.479724 24421 raft_consensus.cc:385] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:30.479882 24421 raft_consensus.cc:740] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48e33bc1e8e94d4887bb64214d0b6c36, State: Initialized, Role: FOLLOWER
I20260810 01:57:30.480195 24187 tablet_service.cc:1511] Processing CreateTablet for tablet 89d9556cc526444299c786de606bb5b1 (DEFAULT_TABLE table=default.table_510ae1b5fcc74db788ffb64925cad250 [id=7d240d10089c47eaa014d59c4ac7f0b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:30.480177 24421 consensus_queue.cc:260] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.480633 24187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89d9556cc526444299c786de606bb5b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:30.481067 24421 ts_tablet_manager.cc:1434] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 01:57:30.482023 24421 tablet_bootstrap.cc:492] T 1844a63489d74f1c919e96501e4bd568 P 48e33bc1e8e94d4887bb64214d0b6c36: Bootstrap starting.
I20260810 01:57:30.483767 24431 ts_tablet_manager.cc:1929] T 0b876159b5cb420391459b6263472051 P 498d352dbea9405199a121e84a4ab926: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 01:57:30.483844 24421 tablet_bootstrap.cc:654] T 1844a63489d74f1c919e96501e4bd568 P 48e33bc1e8e94d4887bb64214d0b6c36: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:30.483914 24431 log.cc:1199] T 0b876159b5cb420391459b6263472051 P 498d352dbea9405199a121e84a4ab926: Deleting WAL directory at /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-2/wal/wals/0b876159b5cb420391459b6263472051
I20260810 01:57:30.484329 24431 ts_tablet_manager.cc:1950] T 0b876159b5cb420391459b6263472051 P 498d352dbea9405199a121e84a4ab926: Deleting consensus metadata
I20260810 01:57:30.484562 23964 tablet_service.cc:1511] Processing CreateTablet for tablet 89d9556cc526444299c786de606bb5b1 (DEFAULT_TABLE table=default.table_510ae1b5fcc74db788ffb64925cad250 [id=7d240d10089c47eaa014d59c4ac7f0b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:30.484973 23964 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89d9556cc526444299c786de606bb5b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:30.485682 24424 tablet_bootstrap.cc:492] T 1844a63489d74f1c919e96501e4bd568 P 50d44445234b4931aaa28141cdaa67ab: No bootstrap required, opened a new log
I20260810 01:57:30.485954 24424 ts_tablet_manager.cc:1403] T 1844a63489d74f1c919e96501e4bd568 P 50d44445234b4931aaa28141cdaa67ab: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.003s
I20260810 01:57:30.486742 24424 raft_consensus.cc:359] T 1844a63489d74f1c919e96501e4bd568 P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.486958 24424 raft_consensus.cc:385] T 1844a63489d74f1c919e96501e4bd568 P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:30.487038 24424 raft_consensus.cc:740] T 1844a63489d74f1c919e96501e4bd568 P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50d44445234b4931aaa28141cdaa67ab, State: Initialized, Role: FOLLOWER
I20260810 01:57:30.487448 24424 consensus_queue.cc:260] T 1844a63489d74f1c919e96501e4bd568 P 50d44445234b4931aaa28141cdaa67ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.491724 24424 ts_tablet_manager.cc:1434] T 1844a63489d74f1c919e96501e4bd568 P 50d44445234b4931aaa28141cdaa67ab: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260810 01:57:30.494590 24424 tablet_bootstrap.cc:492] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 50d44445234b4931aaa28141cdaa67ab: Bootstrap starting.
I20260810 01:57:30.496264 24424 tablet_bootstrap.cc:654] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 50d44445234b4931aaa28141cdaa67ab: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:30.497997 23375 catalog_manager.cc:5050] TS 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): tablet 0b876159b5cb420391459b6263472051 (table default.table_72b6f93a124042e4a090dc194134c75a [id=017756566b72415aa33842f8df2b3e91]) successfully deleted
W20260810 01:57:30.498683 23375 catalog_manager.cc:5017] TS 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): delete failed for tablet b24af70442e44ca4800206b1b5bd7715 because the tablet was not found. No further retry: Not found: Tablet not found: b24af70442e44ca4800206b1b5bd7715
I20260810 01:57:30.502053 24424 tablet_bootstrap.cc:492] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 50d44445234b4931aaa28141cdaa67ab: No bootstrap required, opened a new log
I20260810 01:57:30.502197 24424 ts_tablet_manager.cc:1403] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 50d44445234b4931aaa28141cdaa67ab: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.002s
I20260810 01:57:30.502964 24424 raft_consensus.cc:359] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.503207 24424 raft_consensus.cc:385] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:30.503315 24424 raft_consensus.cc:740] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50d44445234b4931aaa28141cdaa67ab, State: Initialized, Role: FOLLOWER
I20260810 01:57:30.503595 24424 consensus_queue.cc:260] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 50d44445234b4931aaa28141cdaa67ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.506907 24424 ts_tablet_manager.cc:1434] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 50d44445234b4931aaa28141cdaa67ab: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260810 01:57:30.508387 22804 external_mini_cluster.cc:1372] Running /tmp/dist-test-task_qhVVJ/build/asan/bin/kudu
/tmp/dist-test-task_qhVVJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.69.60:36917
--webserver_interface=127.22.69.60
--webserver_port=44467
--builtin_ntp_servers=127.22.69.20:43209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917
--hive_metastore_uris=thrift://127.0.0.1:37513
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260810 01:57:30.514281 24429 ts_tablet_manager.cc:1929] T 0b876159b5cb420391459b6263472051 P 48e33bc1e8e94d4887bb64214d0b6c36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 01:57:30.514451 24429 log.cc:1199] T 0b876159b5cb420391459b6263472051 P 48e33bc1e8e94d4887bb64214d0b6c36: Deleting WAL directory at /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-1/wal/wals/0b876159b5cb420391459b6263472051
I20260810 01:57:30.514875 24429 ts_tablet_manager.cc:1950] T 0b876159b5cb420391459b6263472051 P 48e33bc1e8e94d4887bb64214d0b6c36: Deleting consensus metadata
I20260810 01:57:30.520370 24422 tablet_bootstrap.cc:492] T 1844a63489d74f1c919e96501e4bd568 P 498d352dbea9405199a121e84a4ab926: No bootstrap required, opened a new log
I20260810 01:57:30.520619 24422 ts_tablet_manager.cc:1403] T 1844a63489d74f1c919e96501e4bd568 P 498d352dbea9405199a121e84a4ab926: Time spent bootstrapping tablet: real 0.056s user 0.002s sys 0.003s
I20260810 01:57:30.521880 24422 raft_consensus.cc:359] T 1844a63489d74f1c919e96501e4bd568 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.522159 24422 raft_consensus.cc:385] T 1844a63489d74f1c919e96501e4bd568 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:30.522429 24422 raft_consensus.cc:740] T 1844a63489d74f1c919e96501e4bd568 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 498d352dbea9405199a121e84a4ab926, State: Initialized, Role: FOLLOWER
I20260810 01:57:30.522755 24422 consensus_queue.cc:260] T 1844a63489d74f1c919e96501e4bd568 P 498d352dbea9405199a121e84a4ab926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.523639 24422 ts_tablet_manager.cc:1434] T 1844a63489d74f1c919e96501e4bd568 P 498d352dbea9405199a121e84a4ab926: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260810 01:57:30.523999 24422 tablet_bootstrap.cc:492] T 16db112abfb34d4a9f2ecbee98c39d8b P 498d352dbea9405199a121e84a4ab926: Bootstrap starting.
W20260810 01:57:30.524838 23850 tablet.cc:2406] T 09ed68395e2d44a5b51a22abe2203406 P 50d44445234b4931aaa28141cdaa67ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 01:57:30.525993 24422 tablet_bootstrap.cc:654] T 16db112abfb34d4a9f2ecbee98c39d8b P 498d352dbea9405199a121e84a4ab926: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:30.533278 23366 catalog_manager.cc:5050] TS 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): tablet 0b876159b5cb420391459b6263472051 (table default.table_72b6f93a124042e4a090dc194134c75a [id=017756566b72415aa33842f8df2b3e91]) successfully deleted
I20260810 01:57:30.533663 24429 tablet_replica.cc:333] T b24af70442e44ca4800206b1b5bd7715 P 48e33bc1e8e94d4887bb64214d0b6c36: stopping tablet replica
I20260810 01:57:30.533840 24429 raft_consensus.cc:2243] T b24af70442e44ca4800206b1b5bd7715 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 01:57:30.534021 24429 raft_consensus.cc:2272] T b24af70442e44ca4800206b1b5bd7715 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 01:57:30.534853 24429 ts_tablet_manager.cc:1916] T b24af70442e44ca4800206b1b5bd7715 P 48e33bc1e8e94d4887bb64214d0b6c36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:57:30.538479 24421 tablet_bootstrap.cc:492] T 1844a63489d74f1c919e96501e4bd568 P 48e33bc1e8e94d4887bb64214d0b6c36: No bootstrap required, opened a new log
I20260810 01:57:30.538725 24421 ts_tablet_manager.cc:1403] T 1844a63489d74f1c919e96501e4bd568 P 48e33bc1e8e94d4887bb64214d0b6c36: Time spent bootstrapping tablet: real 0.057s user 0.028s sys 0.025s
I20260810 01:57:30.539649 24421 raft_consensus.cc:359] T 1844a63489d74f1c919e96501e4bd568 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.539963 24421 raft_consensus.cc:385] T 1844a63489d74f1c919e96501e4bd568 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:30.540096 24421 raft_consensus.cc:740] T 1844a63489d74f1c919e96501e4bd568 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48e33bc1e8e94d4887bb64214d0b6c36, State: Initialized, Role: FOLLOWER
I20260810 01:57:30.540438 24421 consensus_queue.cc:260] T 1844a63489d74f1c919e96501e4bd568 P 48e33bc1e8e94d4887bb64214d0b6c36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.542333 24424 tablet_bootstrap.cc:492] T 16db112abfb34d4a9f2ecbee98c39d8b P 50d44445234b4931aaa28141cdaa67ab: Bootstrap starting.
I20260810 01:57:30.543179 24421 ts_tablet_manager.cc:1434] T 1844a63489d74f1c919e96501e4bd568 P 48e33bc1e8e94d4887bb64214d0b6c36: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260810 01:57:30.545644 24421 tablet_bootstrap.cc:492] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 48e33bc1e8e94d4887bb64214d0b6c36: Bootstrap starting.
I20260810 01:57:30.547375 24429 ts_tablet_manager.cc:1929] T b24af70442e44ca4800206b1b5bd7715 P 48e33bc1e8e94d4887bb64214d0b6c36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 01:57:30.547497 24429 log.cc:1199] T b24af70442e44ca4800206b1b5bd7715 P 48e33bc1e8e94d4887bb64214d0b6c36: Deleting WAL directory at /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-1/wal/wals/b24af70442e44ca4800206b1b5bd7715
I20260810 01:57:30.547887 24429 ts_tablet_manager.cc:1950] T b24af70442e44ca4800206b1b5bd7715 P 48e33bc1e8e94d4887bb64214d0b6c36: Deleting consensus metadata
I20260810 01:57:30.548056 24421 tablet_bootstrap.cc:654] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 48e33bc1e8e94d4887bb64214d0b6c36: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:30.548863 23366 catalog_manager.cc:5050] TS 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): tablet b24af70442e44ca4800206b1b5bd7715 (table default.table_c8ad906be0f74210912a57adead21255 [id=4da3c920ea6a48588c9f0c91da00fb69]) successfully deleted
W20260810 01:57:30.549744 24071 tablet.cc:2406] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 01:57:30.553442 24424 tablet_bootstrap.cc:654] T 16db112abfb34d4a9f2ecbee98c39d8b P 50d44445234b4931aaa28141cdaa67ab: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:30.557260 24421 tablet_bootstrap.cc:492] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 48e33bc1e8e94d4887bb64214d0b6c36: No bootstrap required, opened a new log
I20260810 01:57:30.557443 24421 ts_tablet_manager.cc:1403] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 48e33bc1e8e94d4887bb64214d0b6c36: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.006s
I20260810 01:57:30.558233 24421 raft_consensus.cc:359] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.558508 24421 raft_consensus.cc:385] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:30.558652 24421 raft_consensus.cc:740] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48e33bc1e8e94d4887bb64214d0b6c36, State: Initialized, Role: FOLLOWER
I20260810 01:57:30.558962 24421 consensus_queue.cc:260] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 48e33bc1e8e94d4887bb64214d0b6c36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.559278 24424 tablet_bootstrap.cc:492] T 16db112abfb34d4a9f2ecbee98c39d8b P 50d44445234b4931aaa28141cdaa67ab: No bootstrap required, opened a new log
I20260810 01:57:30.559489 24424 ts_tablet_manager.cc:1403] T 16db112abfb34d4a9f2ecbee98c39d8b P 50d44445234b4931aaa28141cdaa67ab: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.001s
I20260810 01:57:30.560493 24424 raft_consensus.cc:359] T 16db112abfb34d4a9f2ecbee98c39d8b P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.560742 24424 raft_consensus.cc:385] T 16db112abfb34d4a9f2ecbee98c39d8b P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:30.560824 24424 raft_consensus.cc:740] T 16db112abfb34d4a9f2ecbee98c39d8b P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50d44445234b4931aaa28141cdaa67ab, State: Initialized, Role: FOLLOWER
I20260810 01:57:30.561527 24421 ts_tablet_manager.cc:1434] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 48e33bc1e8e94d4887bb64214d0b6c36: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260810 01:57:30.563954 24421 tablet_bootstrap.cc:492] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36: Bootstrap starting.
I20260810 01:57:30.564265 24422 tablet_bootstrap.cc:492] T 16db112abfb34d4a9f2ecbee98c39d8b P 498d352dbea9405199a121e84a4ab926: No bootstrap required, opened a new log
I20260810 01:57:30.564452 24422 ts_tablet_manager.cc:1403] T 16db112abfb34d4a9f2ecbee98c39d8b P 498d352dbea9405199a121e84a4ab926: Time spent bootstrapping tablet: real 0.041s user 0.015s sys 0.004s
I20260810 01:57:30.565305 24422 raft_consensus.cc:359] T 16db112abfb34d4a9f2ecbee98c39d8b P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.565536 24422 raft_consensus.cc:385] T 16db112abfb34d4a9f2ecbee98c39d8b P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:30.565632 24422 raft_consensus.cc:740] T 16db112abfb34d4a9f2ecbee98c39d8b P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 498d352dbea9405199a121e84a4ab926, State: Initialized, Role: FOLLOWER
I20260810 01:57:30.565941 24422 consensus_queue.cc:260] T 16db112abfb34d4a9f2ecbee98c39d8b P 498d352dbea9405199a121e84a4ab926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.566324 24430 raft_consensus.cc:493] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:57:30.566387 24421 tablet_bootstrap.cc:654] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:30.566589 24422 ts_tablet_manager.cc:1434] T 16db112abfb34d4a9f2ecbee98c39d8b P 498d352dbea9405199a121e84a4ab926: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260810 01:57:30.566560 24430 raft_consensus.cc:515] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.566955 24422 tablet_bootstrap.cc:492] T 09ed68395e2d44a5b51a22abe2203406 P 498d352dbea9405199a121e84a4ab926: Bootstrap starting.
I20260810 01:57:30.568663 24422 tablet_bootstrap.cc:654] T 09ed68395e2d44a5b51a22abe2203406 P 498d352dbea9405199a121e84a4ab926: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:30.568825 24430 leader_election.cc:290] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 50d44445234b4931aaa28141cdaa67ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
I20260810 01:57:30.574703 24421 tablet_bootstrap.cc:492] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36: No bootstrap required, opened a new log
I20260810 01:57:30.574928 24421 ts_tablet_manager.cc:1403] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.004s
I20260810 01:57:30.575750 24421 raft_consensus.cc:359] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.575995 24421 raft_consensus.cc:385] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:30.576115 24421 raft_consensus.cc:740] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48e33bc1e8e94d4887bb64214d0b6c36, State: Initialized, Role: FOLLOWER
I20260810 01:57:30.576434 24421 consensus_queue.cc:260] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.576799 24422 tablet_bootstrap.cc:492] T 09ed68395e2d44a5b51a22abe2203406 P 498d352dbea9405199a121e84a4ab926: No bootstrap required, opened a new log
I20260810 01:57:30.576951 24422 ts_tablet_manager.cc:1403] T 09ed68395e2d44a5b51a22abe2203406 P 498d352dbea9405199a121e84a4ab926: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.003s
I20260810 01:57:30.577693 24422 raft_consensus.cc:359] T 09ed68395e2d44a5b51a22abe2203406 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.577898 24422 raft_consensus.cc:385] T 09ed68395e2d44a5b51a22abe2203406 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:30.577970 24422 raft_consensus.cc:740] T 09ed68395e2d44a5b51a22abe2203406 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 498d352dbea9405199a121e84a4ab926, State: Initialized, Role: FOLLOWER
I20260810 01:57:30.578296 24422 consensus_queue.cc:260] T 09ed68395e2d44a5b51a22abe2203406 P 498d352dbea9405199a121e84a4ab926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.579020 24421 ts_tablet_manager.cc:1434] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260810 01:57:30.582051 24422 ts_tablet_manager.cc:1434] T 09ed68395e2d44a5b51a22abe2203406 P 498d352dbea9405199a121e84a4ab926: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260810 01:57:30.582345 24422 tablet_bootstrap.cc:492] T e6ef3afe40a847dbb08792578d52401d P 498d352dbea9405199a121e84a4ab926: Bootstrap starting.
I20260810 01:57:30.584211 24422 tablet_bootstrap.cc:654] T e6ef3afe40a847dbb08792578d52401d P 498d352dbea9405199a121e84a4ab926: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:30.593909 24421 tablet_bootstrap.cc:492] T e6ef3afe40a847dbb08792578d52401d P 48e33bc1e8e94d4887bb64214d0b6c36: Bootstrap starting.
I20260810 01:57:30.594893 23998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3db8429f9f4cf5b29d3d1a6cc0651c" candidate_uuid: "50d44445234b4931aaa28141cdaa67ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" is_pre_election: true
I20260810 01:57:30.595450 23998 raft_consensus.cc:2468] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50d44445234b4931aaa28141cdaa67ab in term 0.
I20260810 01:57:30.596423 23679 leader_election.cc:304] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 50d44445234b4931aaa28141cdaa67ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab; no voters:
I20260810 01:57:30.596767 24422 tablet_bootstrap.cc:492] T e6ef3afe40a847dbb08792578d52401d P 498d352dbea9405199a121e84a4ab926: No bootstrap required, opened a new log
I20260810 01:57:30.597098 24422 ts_tablet_manager.cc:1403] T e6ef3afe40a847dbb08792578d52401d P 498d352dbea9405199a121e84a4ab926: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.007s
I20260810 01:57:30.597189 24430 raft_consensus.cc:2804] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 01:57:30.597345 24430 raft_consensus.cc:493] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 01:57:30.597453 24430 raft_consensus.cc:3060] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:30.598717 24422 raft_consensus.cc:359] T e6ef3afe40a847dbb08792578d52401d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.598873 24422 raft_consensus.cc:385] T e6ef3afe40a847dbb08792578d52401d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:30.598946 24422 raft_consensus.cc:740] T e6ef3afe40a847dbb08792578d52401d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 498d352dbea9405199a121e84a4ab926, State: Initialized, Role: FOLLOWER
I20260810 01:57:30.599761 24424 consensus_queue.cc:260] T 16db112abfb34d4a9f2ecbee98c39d8b P 50d44445234b4931aaa28141cdaa67ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.600914 24430 raft_consensus.cc:515] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 50d44445234b4931aaa28141cdaa67ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.601404 24422 consensus_queue.cc:260] T e6ef3afe40a847dbb08792578d52401d P 498d352dbea9405199a121e84a4ab926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.602128 24422 ts_tablet_manager.cc:1434] T e6ef3afe40a847dbb08792578d52401d P 498d352dbea9405199a121e84a4ab926: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260810 01:57:30.602113 23998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3db8429f9f4cf5b29d3d1a6cc0651c" candidate_uuid: "50d44445234b4931aaa28141cdaa67ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48e33bc1e8e94d4887bb64214d0b6c36"
I20260810 01:57:30.602298 23998 raft_consensus.cc:3060] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:30.602437 24422 tablet_bootstrap.cc:492] T 89d9556cc526444299c786de606bb5b1 P 498d352dbea9405199a121e84a4ab926: Bootstrap starting.
I20260810 01:57:30.604194 24422 tablet_bootstrap.cc:654] T 89d9556cc526444299c786de606bb5b1 P 498d352dbea9405199a121e84a4ab926: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:30.605264 24421 tablet_bootstrap.cc:654] T e6ef3afe40a847dbb08792578d52401d P 48e33bc1e8e94d4887bb64214d0b6c36: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:30.605955 23998 raft_consensus.cc:2468] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50d44445234b4931aaa28141cdaa67ab in term 1.
I20260810 01:57:30.606478 23679 leader_election.cc:304] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 50d44445234b4931aaa28141cdaa67ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab; no voters:
I20260810 01:57:30.606982 24430 leader_election.cc:290] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 50d44445234b4931aaa28141cdaa67ab [CANDIDATE]: Term 1 election: Requested vote from peers 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
I20260810 01:57:30.607282 24430 raft_consensus.cc:2804] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 50d44445234b4931aaa28141cdaa67ab [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:57:30.607533 24430 raft_consensus.cc:697] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 50d44445234b4931aaa28141cdaa67ab [term 1 LEADER]: Becoming Leader. State: Replica: 50d44445234b4931aaa28141cdaa67ab, State: Running, Role: LEADER
I20260810 01:57:30.608044 24430 consensus_queue.cc:237] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 50d44445234b4931aaa28141cdaa67ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.608582 24424 ts_tablet_manager.cc:1434] T 16db112abfb34d4a9f2ecbee98c39d8b P 50d44445234b4931aaa28141cdaa67ab: Time spent starting tablet: real 0.049s user 0.003s sys 0.000s
I20260810 01:57:30.616595 24424 tablet_bootstrap.cc:492] T e6ef3afe40a847dbb08792578d52401d P 50d44445234b4931aaa28141cdaa67ab: Bootstrap starting.
I20260810 01:57:30.618798 24424 tablet_bootstrap.cc:654] T e6ef3afe40a847dbb08792578d52401d P 50d44445234b4931aaa28141cdaa67ab: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:30.623823 24222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3db8429f9f4cf5b29d3d1a6cc0651c" candidate_uuid: "50d44445234b4931aaa28141cdaa67ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "498d352dbea9405199a121e84a4ab926" is_pre_election: true
I20260810 01:57:30.624392 24222 raft_consensus.cc:2468] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50d44445234b4931aaa28141cdaa67ab in term 0.
I20260810 01:57:30.625532 24222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3db8429f9f4cf5b29d3d1a6cc0651c" candidate_uuid: "50d44445234b4931aaa28141cdaa67ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "498d352dbea9405199a121e84a4ab926"
I20260810 01:57:30.625699 24422 tablet_bootstrap.cc:492] T 89d9556cc526444299c786de606bb5b1 P 498d352dbea9405199a121e84a4ab926: No bootstrap required, opened a new log
I20260810 01:57:30.625768 24222 raft_consensus.cc:3060] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:30.625885 24422 ts_tablet_manager.cc:1403] T 89d9556cc526444299c786de606bb5b1 P 498d352dbea9405199a121e84a4ab926: Time spent bootstrapping tablet: real 0.024s user 0.000s sys 0.012s
I20260810 01:57:30.626750 24437 raft_consensus.cc:493] T 1844a63489d74f1c919e96501e4bd568 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:57:30.626932 24437 raft_consensus.cc:515] T 1844a63489d74f1c919e96501e4bd568 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.629007 24437 leader_election.cc:290] T 1844a63489d74f1c919e96501e4bd568 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
I20260810 01:57:30.629987 24422 raft_consensus.cc:359] T 89d9556cc526444299c786de606bb5b1 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:30.630223 24422 raft_consensus.cc:385] T 89d9556cc526444299c786de606bb5b1 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:30.630316 24422 raft_consensus.cc:740] T 89d9556cc526444299c786de606bb5b1 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 498d352dbea9405199a121e84a4ab926, State: Initialized, Role: FOLLOWER
I20260810 01:57:30.630581 24422 consensus_queue.cc:260] T 89d9556cc526444299c786de606bb5b1 P 498d352dbea9405199a121e84a4ab926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:30.631182 24422 ts_tablet_manager.cc:1434] T 89d9556cc526444299c786de606bb5b1 P 498d352dbea9405199a121e84a4ab926: Time spent starting tablet: real 0.005s user 0.002s sys 0.001s
I20260810 01:57:30.633119 24438 raft_consensus.cc:493] T 1844a63489d74f1c919e96501e4bd568 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:57:30.633281 24438 raft_consensus.cc:515] T 1844a63489d74f1c919e96501e4bd568 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.634337 24222 raft_consensus.cc:2468] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 498d352dbea9405199a121e84a4ab926 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50d44445234b4931aaa28141cdaa67ab in term 1.
I20260810 01:57:30.635211 24438 leader_election.cc:290] T 1844a63489d74f1c919e96501e4bd568 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
I20260810 01:57:30.636057 24421 tablet_bootstrap.cc:492] T e6ef3afe40a847dbb08792578d52401d P 48e33bc1e8e94d4887bb64214d0b6c36: No bootstrap required, opened a new log
I20260810 01:57:30.636211 24421 ts_tablet_manager.cc:1403] T e6ef3afe40a847dbb08792578d52401d P 48e33bc1e8e94d4887bb64214d0b6c36: Time spent bootstrapping tablet: real 0.056s user 0.006s sys 0.012s
I20260810 01:57:30.637130 24421 raft_consensus.cc:359] T e6ef3afe40a847dbb08792578d52401d P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.637370 24421 raft_consensus.cc:385] T e6ef3afe40a847dbb08792578d52401d P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:30.637475 24421 raft_consensus.cc:740] T e6ef3afe40a847dbb08792578d52401d P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48e33bc1e8e94d4887bb64214d0b6c36, State: Initialized, Role: FOLLOWER
I20260810 01:57:30.637722 24421 consensus_queue.cc:260] T e6ef3afe40a847dbb08792578d52401d P 48e33bc1e8e94d4887bb64214d0b6c36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.639853 24421 ts_tablet_manager.cc:1434] T e6ef3afe40a847dbb08792578d52401d P 48e33bc1e8e94d4887bb64214d0b6c36: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260810 01:57:30.640177 24421 tablet_bootstrap.cc:492] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36: Bootstrap starting.
I20260810 01:57:30.643630 24421 tablet_bootstrap.cc:654] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:30.644141 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1844a63489d74f1c919e96501e4bd568" candidate_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50d44445234b4931aaa28141cdaa67ab" is_pre_election: true
I20260810 01:57:30.644556 23776 raft_consensus.cc:2468] T 1844a63489d74f1c919e96501e4bd568 P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48e33bc1e8e94d4887bb64214d0b6c36 in term 0.
I20260810 01:57:30.645998 23889 leader_election.cc:304] T 1844a63489d74f1c919e96501e4bd568 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab; no voters:
I20260810 01:57:30.646404 24437 raft_consensus.cc:2804] T 1844a63489d74f1c919e96501e4bd568 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 01:57:30.646551 24437 raft_consensus.cc:493] T 1844a63489d74f1c919e96501e4bd568 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 01:57:30.646680 24437 raft_consensus.cc:3060] T 1844a63489d74f1c919e96501e4bd568 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:30.647095 24222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1844a63489d74f1c919e96501e4bd568" candidate_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "498d352dbea9405199a121e84a4ab926" is_pre_election: true
I20260810 01:57:30.647306 24222 raft_consensus.cc:2468] T 1844a63489d74f1c919e96501e4bd568 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48e33bc1e8e94d4887bb64214d0b6c36 in term 0.
I20260810 01:57:30.649983 24437 raft_consensus.cc:515] T 1844a63489d74f1c919e96501e4bd568 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.651044 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1844a63489d74f1c919e96501e4bd568" candidate_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50d44445234b4931aaa28141cdaa67ab"
I20260810 01:57:30.651276 23776 raft_consensus.cc:3060] T 1844a63489d74f1c919e96501e4bd568 P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:30.654616 23776 raft_consensus.cc:2468] T 1844a63489d74f1c919e96501e4bd568 P 50d44445234b4931aaa28141cdaa67ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48e33bc1e8e94d4887bb64214d0b6c36 in term 1.
I20260810 01:57:30.655500 23889 leader_election.cc:304] T 1844a63489d74f1c919e96501e4bd568 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab; no voters:
I20260810 01:57:30.657135 24222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1844a63489d74f1c919e96501e4bd568" candidate_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "498d352dbea9405199a121e84a4ab926"
I20260810 01:57:30.657363 24222 raft_consensus.cc:3060] T 1844a63489d74f1c919e96501e4bd568 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:30.659320 24437 leader_election.cc:290] T 1844a63489d74f1c919e96501e4bd568 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 1 election: Requested vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
I20260810 01:57:30.659567 24437 raft_consensus.cc:2804] T 1844a63489d74f1c919e96501e4bd568 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:57:30.659711 24437 raft_consensus.cc:697] T 1844a63489d74f1c919e96501e4bd568 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 LEADER]: Becoming Leader. State: Replica: 48e33bc1e8e94d4887bb64214d0b6c36, State: Running, Role: LEADER
I20260810 01:57:30.660140 24437 consensus_queue.cc:237] T 1844a63489d74f1c919e96501e4bd568 P 48e33bc1e8e94d4887bb64214d0b6c36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.660713 24222 raft_consensus.cc:2468] T 1844a63489d74f1c919e96501e4bd568 P 498d352dbea9405199a121e84a4ab926 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48e33bc1e8e94d4887bb64214d0b6c36 in term 1.
I20260810 01:57:30.671633 23998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1844a63489d74f1c919e96501e4bd568" candidate_uuid: "498d352dbea9405199a121e84a4ab926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" is_pre_election: true
I20260810 01:57:30.672451 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1844a63489d74f1c919e96501e4bd568" candidate_uuid: "498d352dbea9405199a121e84a4ab926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50d44445234b4931aaa28141cdaa67ab" is_pre_election: true
I20260810 01:57:30.672813 23776 raft_consensus.cc:2393] T 1844a63489d74f1c919e96501e4bd568 P 50d44445234b4931aaa28141cdaa67ab [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 498d352dbea9405199a121e84a4ab926 in current term 1: Already voted for candidate 48e33bc1e8e94d4887bb64214d0b6c36 in this term.
I20260810 01:57:30.674652 24109 leader_election.cc:304] T 1844a63489d74f1c919e96501e4bd568 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:57:30.678373 24438 raft_consensus.cc:2749] T 1844a63489d74f1c919e96501e4bd568 P 498d352dbea9405199a121e84a4ab926 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:57:30.692720 24421 tablet_bootstrap.cc:492] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36: No bootstrap required, opened a new log
I20260810 01:57:30.692930 24421 ts_tablet_manager.cc:1403] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36: Time spent bootstrapping tablet: real 0.053s user 0.011s sys 0.011s
I20260810 01:57:30.693069 24424 tablet_bootstrap.cc:492] T e6ef3afe40a847dbb08792578d52401d P 50d44445234b4931aaa28141cdaa67ab: No bootstrap required, opened a new log
I20260810 01:57:30.693352 24424 ts_tablet_manager.cc:1403] T e6ef3afe40a847dbb08792578d52401d P 50d44445234b4931aaa28141cdaa67ab: Time spent bootstrapping tablet: real 0.077s user 0.002s sys 0.002s
I20260810 01:57:30.693794 24421 raft_consensus.cc:359] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:30.694020 24421 raft_consensus.cc:385] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:30.694128 24421 raft_consensus.cc:740] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48e33bc1e8e94d4887bb64214d0b6c36, State: Initialized, Role: FOLLOWER
I20260810 01:57:30.694402 24424 raft_consensus.cc:359] T e6ef3afe40a847dbb08792578d52401d P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.694456 24421 consensus_queue.cc:260] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:30.694680 24424 raft_consensus.cc:385] T e6ef3afe40a847dbb08792578d52401d P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:30.694847 24424 raft_consensus.cc:740] T e6ef3afe40a847dbb08792578d52401d P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50d44445234b4931aaa28141cdaa67ab, State: Initialized, Role: FOLLOWER
I20260810 01:57:30.695194 24424 consensus_queue.cc:260] T e6ef3afe40a847dbb08792578d52401d P 50d44445234b4931aaa28141cdaa67ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.696887 24421 ts_tablet_manager.cc:1434] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260810 01:57:30.697409 24424 ts_tablet_manager.cc:1434] T e6ef3afe40a847dbb08792578d52401d P 50d44445234b4931aaa28141cdaa67ab: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260810 01:57:30.697777 24424 tablet_bootstrap.cc:492] T 89d9556cc526444299c786de606bb5b1 P 50d44445234b4931aaa28141cdaa67ab: Bootstrap starting.
I20260810 01:57:30.699512 24438 raft_consensus.cc:493] T e6ef3afe40a847dbb08792578d52401d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:57:30.699681 24438 raft_consensus.cc:515] T e6ef3afe40a847dbb08792578d52401d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.700300 24438 leader_election.cc:290] T e6ef3afe40a847dbb08792578d52401d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
I20260810 01:57:30.700980 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6ef3afe40a847dbb08792578d52401d" candidate_uuid: "498d352dbea9405199a121e84a4ab926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50d44445234b4931aaa28141cdaa67ab" is_pre_election: true
I20260810 01:57:30.701057 23998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6ef3afe40a847dbb08792578d52401d" candidate_uuid: "498d352dbea9405199a121e84a4ab926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" is_pre_election: true
I20260810 01:57:30.701210 23776 raft_consensus.cc:2468] T e6ef3afe40a847dbb08792578d52401d P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 498d352dbea9405199a121e84a4ab926 in term 0.
I20260810 01:57:30.701228 23998 raft_consensus.cc:2468] T e6ef3afe40a847dbb08792578d52401d P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 498d352dbea9405199a121e84a4ab926 in term 0.
I20260810 01:57:30.701772 24109 leader_election.cc:304] T e6ef3afe40a847dbb08792578d52401d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36, 498d352dbea9405199a121e84a4ab926; no voters:
I20260810 01:57:30.702054 24438 raft_consensus.cc:2804] T e6ef3afe40a847dbb08792578d52401d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 01:57:30.702179 24438 raft_consensus.cc:493] T e6ef3afe40a847dbb08792578d52401d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 01:57:30.702310 24438 raft_consensus.cc:3060] T e6ef3afe40a847dbb08792578d52401d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:30.703600 24424 tablet_bootstrap.cc:654] T 89d9556cc526444299c786de606bb5b1 P 50d44445234b4931aaa28141cdaa67ab: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:30.705912 24438 raft_consensus.cc:515] T e6ef3afe40a847dbb08792578d52401d P 498d352dbea9405199a121e84a4ab926 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.706734 24438 leader_election.cc:290] T e6ef3afe40a847dbb08792578d52401d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 election: Requested vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
I20260810 01:57:30.707023 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6ef3afe40a847dbb08792578d52401d" candidate_uuid: "498d352dbea9405199a121e84a4ab926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50d44445234b4931aaa28141cdaa67ab"
I20260810 01:57:30.707257 23776 raft_consensus.cc:3060] T e6ef3afe40a847dbb08792578d52401d P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:30.709517 23776 raft_consensus.cc:2468] T e6ef3afe40a847dbb08792578d52401d P 50d44445234b4931aaa28141cdaa67ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 498d352dbea9405199a121e84a4ab926 in term 1.
I20260810 01:57:30.710255 23998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6ef3afe40a847dbb08792578d52401d" candidate_uuid: "498d352dbea9405199a121e84a4ab926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48e33bc1e8e94d4887bb64214d0b6c36"
I20260810 01:57:30.710454 23998 raft_consensus.cc:3060] T e6ef3afe40a847dbb08792578d52401d P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:30.712807 23998 raft_consensus.cc:2468] T e6ef3afe40a847dbb08792578d52401d P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 498d352dbea9405199a121e84a4ab926 in term 1.
I20260810 01:57:30.710040 24109 leader_election.cc:304] T e6ef3afe40a847dbb08792578d52401d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab; no voters:
I20260810 01:57:30.713335 24438 raft_consensus.cc:2804] T e6ef3afe40a847dbb08792578d52401d P 498d352dbea9405199a121e84a4ab926 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:57:30.722792 24438 raft_consensus.cc:697] T e6ef3afe40a847dbb08792578d52401d P 498d352dbea9405199a121e84a4ab926 [term 1 LEADER]: Becoming Leader. State: Replica: 498d352dbea9405199a121e84a4ab926, State: Running, Role: LEADER
I20260810 01:57:30.723486 24438 consensus_queue.cc:237] T e6ef3afe40a847dbb08792578d52401d P 498d352dbea9405199a121e84a4ab926 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.733875 24424 tablet_bootstrap.cc:492] T 89d9556cc526444299c786de606bb5b1 P 50d44445234b4931aaa28141cdaa67ab: No bootstrap required, opened a new log
I20260810 01:57:30.734088 24424 ts_tablet_manager.cc:1403] T 89d9556cc526444299c786de606bb5b1 P 50d44445234b4931aaa28141cdaa67ab: Time spent bootstrapping tablet: real 0.036s user 0.004s sys 0.001s
I20260810 01:57:30.734737 24424 raft_consensus.cc:359] T 89d9556cc526444299c786de606bb5b1 P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:30.734974 24424 raft_consensus.cc:385] T 89d9556cc526444299c786de606bb5b1 P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:30.735088 24424 raft_consensus.cc:740] T 89d9556cc526444299c786de606bb5b1 P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50d44445234b4931aaa28141cdaa67ab, State: Initialized, Role: FOLLOWER
I20260810 01:57:30.735374 24424 consensus_queue.cc:260] T 89d9556cc526444299c786de606bb5b1 P 50d44445234b4931aaa28141cdaa67ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:30.735908 24424 ts_tablet_manager.cc:1434] T 89d9556cc526444299c786de606bb5b1 P 50d44445234b4931aaa28141cdaa67ab: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 01:57:30.771178 24438 raft_consensus.cc:493] T 09ed68395e2d44a5b51a22abe2203406 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:57:30.771515 24438 raft_consensus.cc:515] T 09ed68395e2d44a5b51a22abe2203406 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.772372 24438 leader_election.cc:290] T 09ed68395e2d44a5b51a22abe2203406 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
I20260810 01:57:30.773548 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09ed68395e2d44a5b51a22abe2203406" candidate_uuid: "498d352dbea9405199a121e84a4ab926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50d44445234b4931aaa28141cdaa67ab" is_pre_election: true
I20260810 01:57:30.773800 23776 raft_consensus.cc:2468] T 09ed68395e2d44a5b51a22abe2203406 P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 498d352dbea9405199a121e84a4ab926 in term 0.
I20260810 01:57:30.774246 23998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09ed68395e2d44a5b51a22abe2203406" candidate_uuid: "498d352dbea9405199a121e84a4ab926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" is_pre_election: true
I20260810 01:57:30.774495 23998 raft_consensus.cc:2468] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 498d352dbea9405199a121e84a4ab926 in term 0.
I20260810 01:57:30.775736 24109 leader_election.cc:304] T 09ed68395e2d44a5b51a22abe2203406 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36, 498d352dbea9405199a121e84a4ab926; no voters:
I20260810 01:57:30.776314 24438 raft_consensus.cc:2804] T 09ed68395e2d44a5b51a22abe2203406 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 01:57:30.776412 24438 raft_consensus.cc:493] T 09ed68395e2d44a5b51a22abe2203406 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 01:57:30.776520 24438 raft_consensus.cc:3060] T 09ed68395e2d44a5b51a22abe2203406 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:30.779641 24438 raft_consensus.cc:515] T 09ed68395e2d44a5b51a22abe2203406 P 498d352dbea9405199a121e84a4ab926 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.780395 24438 leader_election.cc:290] T 09ed68395e2d44a5b51a22abe2203406 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 election: Requested vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
I20260810 01:57:30.781373 23998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09ed68395e2d44a5b51a22abe2203406" candidate_uuid: "498d352dbea9405199a121e84a4ab926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48e33bc1e8e94d4887bb64214d0b6c36"
I20260810 01:57:30.781589 23998 raft_consensus.cc:3060] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:30.781589 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09ed68395e2d44a5b51a22abe2203406" candidate_uuid: "498d352dbea9405199a121e84a4ab926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50d44445234b4931aaa28141cdaa67ab"
I20260810 01:57:30.781764 23776 raft_consensus.cc:3060] T 09ed68395e2d44a5b51a22abe2203406 P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:30.784787 23998 raft_consensus.cc:2468] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 498d352dbea9405199a121e84a4ab926 in term 1.
I20260810 01:57:30.784803 23776 raft_consensus.cc:2468] T 09ed68395e2d44a5b51a22abe2203406 P 50d44445234b4931aaa28141cdaa67ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 498d352dbea9405199a121e84a4ab926 in term 1.
I20260810 01:57:30.785286 24109 leader_election.cc:304] T 09ed68395e2d44a5b51a22abe2203406 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab; no voters:
I20260810 01:57:30.785684 24438 raft_consensus.cc:2804] T 09ed68395e2d44a5b51a22abe2203406 P 498d352dbea9405199a121e84a4ab926 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:57:30.785902 24438 raft_consensus.cc:697] T 09ed68395e2d44a5b51a22abe2203406 P 498d352dbea9405199a121e84a4ab926 [term 1 LEADER]: Becoming Leader. State: Replica: 498d352dbea9405199a121e84a4ab926, State: Running, Role: LEADER
I20260810 01:57:30.786224 24438 consensus_queue.cc:237] T 09ed68395e2d44a5b51a22abe2203406 P 498d352dbea9405199a121e84a4ab926 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
01:57:30.801 [HMS - ERROR - pool-6-thread-3] (RetryingHMSHandler.java:201) AlreadyExistsException(message:Table hive.default.table_4fb63c687c11493ca7c0b76735f137e9 already exists)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1839)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1786)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_with_environment_context(HiveMetaStore.java:2035)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:569)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
at jdk.proxy2/jdk.proxy2.$Proxy27.create_table_with_environment_context(Unknown Source)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:14800)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:14784)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:104)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
W20260810 01:57:30.813452 23385 catalog_manager.cc:2196] Already present: failed to create HMS catalog entry for table [id=60645a17aa334d9c8bce17dfbc5cdf83]: failed to create Hive MetaStore table: TException - service has thrown: AlreadyExistsException(message=Table hive.default.table_4fb63c687c11493ca7c0b76735f137e9 already exists)
I20260810 01:57:30.813978 23385 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:45748 (request call id 8) took 1072 ms. Trace:
I20260810 01:57:30.814122 23385 rpcz_store.cc:276] 0810 01:57:29.741802 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 01:57:29.743270 (+ 1468us) service_pool.cc:224] Handling call
0810 01:57:29.769294 (+ 26024us) catalog_manager.cc:2127] Acquired catalog manager lock
0810 01:57:29.769763 (+ 469us) catalog_manager.cc:2173] Created new table and tablet info
0810 01:57:30.813929 (+1044166us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":686222,"Hive Metastore.run_cpu_time_us":2348,"Hive Metastore.run_wall_time_us":356861}
I20260810 01:57:30.828367 23385 catalog_manager.cc:5719] T 09ed68395e2d44a5b51a22abe2203406 P 498d352dbea9405199a121e84a4ab926 reported cstate change: term changed from 0 to 1, leader changed from <none> to 498d352dbea9405199a121e84a4ab926 (127.22.69.3). New cstate: current_term: 1 leader_uuid: "498d352dbea9405199a121e84a4ab926" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } health_report { overall_health: HEALTHY } } }
I20260810 01:57:30.829605 23385 catalog_manager.cc:5719] T e6ef3afe40a847dbb08792578d52401d P 498d352dbea9405199a121e84a4ab926 reported cstate change: term changed from 0 to 1, leader changed from <none> to 498d352dbea9405199a121e84a4ab926 (127.22.69.3). New cstate: current_term: 1 leader_uuid: "498d352dbea9405199a121e84a4ab926" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } health_report { overall_health: HEALTHY } } }
I20260810 01:57:30.841169 23385 catalog_manager.cc:5719] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 50d44445234b4931aaa28141cdaa67ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 50d44445234b4931aaa28141cdaa67ab (127.22.69.1). New cstate: current_term: 1 leader_uuid: "50d44445234b4931aaa28141cdaa67ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } health_report { overall_health: UNKNOWN } } }
I20260810 01:57:30.856504 23385 catalog_manager.cc:5719] T 1844a63489d74f1c919e96501e4bd568 P 48e33bc1e8e94d4887bb64214d0b6c36 reported cstate change: term changed from 0 to 1, leader changed from <none> to 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2). New cstate: current_term: 1 leader_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } health_report { overall_health: UNKNOWN } } }
I20260810 01:57:30.872347 23385 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45748:
name: "default.Table_9175c2bd9b0542f2a6b95eae8a882b7a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:30.879849 23387 catalog_manager.cc:2200] created HMS catalog entry for table [id=5edf9cb336414ae3bd41e9f5ca41fcca]
I20260810 01:57:30.891276 23387 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:45748 (request call id 6) took 1149 ms. Trace:
I20260810 01:57:30.891438 23387 rpcz_store.cc:276] 0810 01:57:29.741536 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 01:57:29.742861 (+ 1325us) service_pool.cc:224] Handling call
0810 01:57:29.770926 (+ 28065us) catalog_manager.cc:2127] Acquired catalog manager lock
0810 01:57:29.771535 (+ 609us) catalog_manager.cc:2173] Created new table and tablet info
0810 01:57:30.879794 (+1108259us) catalog_manager.cc:2199] Created new table in HMS catalog
0810 01:57:30.886167 (+ 6373us) catalog_manager.cc:2233] Wrote table and tablets to system table
0810 01:57:30.886302 (+ 135us) catalog_manager.cc:2267] Inserted table and tablets into CatalogManager maps
0810 01:57:30.891228 (+ 4926us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 01:57:30.880649 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 01:57:30.880766 (+ 117us) write_op.cc:432] Acquiring schema lock in shared mode
0810 01:57:30.880771 (+ 5us) write_op.cc:435] Acquired schema lock
0810 01:57:30.880773 (+ 2us) tablet.cc:662] Decoding operations
0810 01:57:30.880855 (+ 82us) write_op.cc:620] Acquiring the partition lock for write op
0810 01:57:30.880869 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0810 01:57:30.880874 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0810 01:57:30.880965 (+ 91us) tablet.cc:701] Row locks acquired
0810 01:57:30.880967 (+ 2us) write_op.cc:260] PREPARE: finished
0810 01:57:30.881003 (+ 36us) write_op.cc:270] Start()
0810 01:57:30.881034 (+ 31us) write_op.cc:276] Timestamp: P: 1786327050880950 usec, L: 0
0810 01:57:30.881037 (+ 3us) op_driver.cc:348] REPLICATION: starting
0810 01:57:30.881200 (+ 163us) log.cc:844] Serialized 432 byte log entry
0810 01:57:30.884819 (+ 3619us) op_driver.cc:464] REPLICATION: finished
0810 01:57:30.884966 (+ 147us) write_op.cc:301] APPLY: starting
0810 01:57:30.884985 (+ 19us) tablet.cc:1367] starting BulkCheckPresence
0810 01:57:30.885031 (+ 46us) tablet.cc:1370] finished BulkCheckPresence
0810 01:57:30.885065 (+ 34us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 01:57:30.885176 (+ 111us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 01:57:30.885209 (+ 33us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0810 01:57:30.885216 (+ 7us) write_op.cc:312] APPLY: finished
0810 01:57:30.885292 (+ 76us) log.cc:844] Serialized 28 byte log entry
0810 01:57:30.885369 (+ 77us) write_op.cc:489] Releasing partition, row and schema locks
0810 01:57:30.885389 (+ 20us) write_op.cc:454] Released schema lock
0810 01:57:30.885395 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0810 01:57:30.885469 (+ 74us) write_op.cc:489] Releasing partition, row and schema locks
0810 01:57:30.885473 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":1041363,"Hive Metastore.run_cpu_time_us":325,"Hive Metastore.run_wall_time_us":66456,"child_traces":[["op",{"apply.queue_time_us":95,"apply.run_cpu_time_us":498,"apply.run_wall_time_us":497,"num_ops":2,"prepare.queue_time_us":148,"prepare.run_cpu_time_us":679,"prepare.run_wall_time_us":679,"raft.queue_time_us":1044,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":232,"replication_time_us":3751}]]}
I20260810 01:57:30.892386 23387 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45748:
table { table_name: "default.Table_983bf2e604df4fdbac75c5aca292f736" } new_table_name: "default.Table_df285e759e764cdeb38de2aad0d1873d" modify_external_catalogs: true
I20260810 01:57:30.899427 23746 tablet_service.cc:1511] Processing CreateTablet for tablet 4579a90be0b6409781fd9e470324192c (DEFAULT_TABLE table=default.table_22726fd1a11f409f9f5287ab16d8e551 [id=5edf9cb336414ae3bd41e9f5ca41fcca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:30.899696 23964 tablet_service.cc:1511] Processing CreateTablet for tablet 4579a90be0b6409781fd9e470324192c (DEFAULT_TABLE table=default.table_22726fd1a11f409f9f5287ab16d8e551 [id=5edf9cb336414ae3bd41e9f5ca41fcca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:30.899991 23746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4579a90be0b6409781fd9e470324192c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:30.900197 23964 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4579a90be0b6409781fd9e470324192c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:30.906833 24187 tablet_service.cc:1511] Processing CreateTablet for tablet 4579a90be0b6409781fd9e470324192c (DEFAULT_TABLE table=default.table_22726fd1a11f409f9f5287ab16d8e551 [id=5edf9cb336414ae3bd41e9f5ca41fcca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:30.907311 24187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4579a90be0b6409781fd9e470324192c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:30.910481 24421 tablet_bootstrap.cc:492] T 4579a90be0b6409781fd9e470324192c P 48e33bc1e8e94d4887bb64214d0b6c36: Bootstrap starting.
I20260810 01:57:30.912315 24421 tablet_bootstrap.cc:654] T 4579a90be0b6409781fd9e470324192c P 48e33bc1e8e94d4887bb64214d0b6c36: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:30.916965 24422 tablet_bootstrap.cc:492] T 4579a90be0b6409781fd9e470324192c P 498d352dbea9405199a121e84a4ab926: Bootstrap starting.
I20260810 01:57:30.918678 24422 tablet_bootstrap.cc:654] T 4579a90be0b6409781fd9e470324192c P 498d352dbea9405199a121e84a4ab926: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:30.919344 24424 tablet_bootstrap.cc:492] T 4579a90be0b6409781fd9e470324192c P 50d44445234b4931aaa28141cdaa67ab: Bootstrap starting.
I20260810 01:57:30.921082 24424 tablet_bootstrap.cc:654] T 4579a90be0b6409781fd9e470324192c P 50d44445234b4931aaa28141cdaa67ab: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:30.928586 24422 tablet_bootstrap.cc:492] T 4579a90be0b6409781fd9e470324192c P 498d352dbea9405199a121e84a4ab926: No bootstrap required, opened a new log
I20260810 01:57:30.928751 24422 ts_tablet_manager.cc:1403] T 4579a90be0b6409781fd9e470324192c P 498d352dbea9405199a121e84a4ab926: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.001s
I20260810 01:57:30.929581 24422 raft_consensus.cc:359] T 4579a90be0b6409781fd9e470324192c P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.929773 24422 raft_consensus.cc:385] T 4579a90be0b6409781fd9e470324192c P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:30.929876 24422 raft_consensus.cc:740] T 4579a90be0b6409781fd9e470324192c P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 498d352dbea9405199a121e84a4ab926, State: Initialized, Role: FOLLOWER
I20260810 01:57:30.930173 24422 consensus_queue.cc:260] T 4579a90be0b6409781fd9e470324192c P 498d352dbea9405199a121e84a4ab926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.930723 24422 ts_tablet_manager.cc:1434] T 4579a90be0b6409781fd9e470324192c P 498d352dbea9405199a121e84a4ab926: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260810 01:57:30.942191 23392 catalog_manager.cc:2200] created HMS catalog entry for table [id=ef2e836fffc64aa6a8ddc0425f39e317]
I20260810 01:57:30.948266 23392 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:45748 (request call id 2) took 1207 ms. Trace:
I20260810 01:57:30.948410 23392 rpcz_store.cc:276] 0810 01:57:29.740942 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 01:57:29.754765 (+ 13823us) service_pool.cc:224] Handling call
0810 01:57:29.772361 (+ 17596us) catalog_manager.cc:2127] Acquired catalog manager lock
0810 01:57:29.772939 (+ 578us) catalog_manager.cc:2173] Created new table and tablet info
0810 01:57:30.942140 (+1169201us) catalog_manager.cc:2199] Created new table in HMS catalog
0810 01:57:30.947877 (+ 5737us) catalog_manager.cc:2233] Wrote table and tablets to system table
0810 01:57:30.948028 (+ 151us) catalog_manager.cc:2267] Inserted table and tablets into CatalogManager maps
0810 01:57:30.948170 (+ 142us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 01:57:30.943016 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 01:57:30.943196 (+ 180us) write_op.cc:432] Acquiring schema lock in shared mode
0810 01:57:30.943203 (+ 7us) write_op.cc:435] Acquired schema lock
0810 01:57:30.943206 (+ 3us) tablet.cc:662] Decoding operations
0810 01:57:30.943307 (+ 101us) write_op.cc:620] Acquiring the partition lock for write op
0810 01:57:30.943322 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0810 01:57:30.943327 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0810 01:57:30.943430 (+ 103us) tablet.cc:701] Row locks acquired
0810 01:57:30.943433 (+ 3us) write_op.cc:260] PREPARE: finished
0810 01:57:30.943479 (+ 46us) write_op.cc:270] Start()
0810 01:57:30.943516 (+ 37us) write_op.cc:276] Timestamp: P: 1786327050943418 usec, L: 0
0810 01:57:30.943519 (+ 3us) op_driver.cc:348] REPLICATION: starting
0810 01:57:30.943735 (+ 216us) log.cc:844] Serialized 432 byte log entry
0810 01:57:30.947165 (+ 3430us) op_driver.cc:464] REPLICATION: finished
0810 01:57:30.947309 (+ 144us) write_op.cc:301] APPLY: starting
0810 01:57:30.947328 (+ 19us) tablet.cc:1367] starting BulkCheckPresence
0810 01:57:30.947392 (+ 64us) tablet.cc:1370] finished BulkCheckPresence
0810 01:57:30.947420 (+ 28us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 01:57:30.947534 (+ 114us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 01:57:30.947571 (+ 37us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0810 01:57:30.947579 (+ 8us) write_op.cc:312] APPLY: finished
0810 01:57:30.947659 (+ 80us) log.cc:844] Serialized 28 byte log entry
0810 01:57:30.947728 (+ 69us) write_op.cc:489] Releasing partition, row and schema locks
0810 01:57:30.947748 (+ 20us) write_op.cc:454] Released schema lock
0810 01:57:30.947754 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0810 01:57:30.947836 (+ 82us) write_op.cc:489] Releasing partition, row and schema locks
0810 01:57:30.947840 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":1106461,"Hive Metastore.run_cpu_time_us":309,"Hive Metastore.run_wall_time_us":62499,"child_traces":[["op",{"apply.queue_time_us":92,"apply.run_cpu_time_us":517,"apply.run_wall_time_us":516,"num_ops":2,"prepare.queue_time_us":57,"prepare.run_cpu_time_us":857,"prepare.run_wall_time_us":858,"raft.queue_time_us":973,"raft.run_cpu_time_us":287,"raft.run_wall_time_us":288,"replication_time_us":3616}]]}
I20260810 01:57:30.949671 24421 tablet_bootstrap.cc:492] T 4579a90be0b6409781fd9e470324192c P 48e33bc1e8e94d4887bb64214d0b6c36: No bootstrap required, opened a new log
I20260810 01:57:30.949843 24421 ts_tablet_manager.cc:1403] T 4579a90be0b6409781fd9e470324192c P 48e33bc1e8e94d4887bb64214d0b6c36: Time spent bootstrapping tablet: real 0.040s user 0.006s sys 0.006s
I20260810 01:57:30.950798 24421 raft_consensus.cc:359] T 4579a90be0b6409781fd9e470324192c P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.950999 24421 raft_consensus.cc:385] T 4579a90be0b6409781fd9e470324192c P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:30.951112 24421 raft_consensus.cc:740] T 4579a90be0b6409781fd9e470324192c P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48e33bc1e8e94d4887bb64214d0b6c36, State: Initialized, Role: FOLLOWER
I20260810 01:57:30.951380 24421 consensus_queue.cc:260] T 4579a90be0b6409781fd9e470324192c P 48e33bc1e8e94d4887bb64214d0b6c36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.952675 24424 tablet_bootstrap.cc:492] T 4579a90be0b6409781fd9e470324192c P 50d44445234b4931aaa28141cdaa67ab: No bootstrap required, opened a new log
I20260810 01:57:30.952915 24424 ts_tablet_manager.cc:1403] T 4579a90be0b6409781fd9e470324192c P 50d44445234b4931aaa28141cdaa67ab: Time spent bootstrapping tablet: real 0.034s user 0.004s sys 0.006s
I20260810 01:57:30.954295 24424 raft_consensus.cc:359] T 4579a90be0b6409781fd9e470324192c P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.954560 24424 raft_consensus.cc:385] T 4579a90be0b6409781fd9e470324192c P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:30.954711 24424 raft_consensus.cc:740] T 4579a90be0b6409781fd9e470324192c P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50d44445234b4931aaa28141cdaa67ab, State: Initialized, Role: FOLLOWER
I20260810 01:57:30.955010 24424 consensus_queue.cc:260] T 4579a90be0b6409781fd9e470324192c P 50d44445234b4931aaa28141cdaa67ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.955686 24424 ts_tablet_manager.cc:1434] T 4579a90be0b6409781fd9e470324192c P 50d44445234b4931aaa28141cdaa67ab: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260810 01:57:30.958706 24421 ts_tablet_manager.cc:1434] T 4579a90be0b6409781fd9e470324192c P 48e33bc1e8e94d4887bb64214d0b6c36: Time spent starting tablet: real 0.008s user 0.000s sys 0.003s
I20260810 01:57:30.959319 23746 tablet_service.cc:1511] Processing CreateTablet for tablet a48e9f780b4e49abafaadbd78ab8711a (DEFAULT_TABLE table=default.table_57697192c9bf44969c2b200e94865a3e [id=ef2e836fffc64aa6a8ddc0425f39e317]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:30.959839 23746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a48e9f780b4e49abafaadbd78ab8711a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:30.960088 23964 tablet_service.cc:1511] Processing CreateTablet for tablet a48e9f780b4e49abafaadbd78ab8711a (DEFAULT_TABLE table=default.table_57697192c9bf44969c2b200e94865a3e [id=ef2e836fffc64aa6a8ddc0425f39e317]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:30.960469 23964 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a48e9f780b4e49abafaadbd78ab8711a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:30.961408 23392 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45748:
name: "default.Table_7e5efb367a02411aac568a75dd39b274"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:30.966399 24424 tablet_bootstrap.cc:492] T a48e9f780b4e49abafaadbd78ab8711a P 50d44445234b4931aaa28141cdaa67ab: Bootstrap starting.
I20260810 01:57:30.966599 24187 tablet_service.cc:1511] Processing CreateTablet for tablet a48e9f780b4e49abafaadbd78ab8711a (DEFAULT_TABLE table=default.table_57697192c9bf44969c2b200e94865a3e [id=ef2e836fffc64aa6a8ddc0425f39e317]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:30.967078 24187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a48e9f780b4e49abafaadbd78ab8711a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:30.968024 24424 tablet_bootstrap.cc:654] T a48e9f780b4e49abafaadbd78ab8711a P 50d44445234b4931aaa28141cdaa67ab: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:30.969590 24424 tablet_bootstrap.cc:492] T a48e9f780b4e49abafaadbd78ab8711a P 50d44445234b4931aaa28141cdaa67ab: No bootstrap required, opened a new log
I20260810 01:57:30.969767 24424 ts_tablet_manager.cc:1403] T a48e9f780b4e49abafaadbd78ab8711a P 50d44445234b4931aaa28141cdaa67ab: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.003s
I20260810 01:57:30.970347 24424 raft_consensus.cc:359] T a48e9f780b4e49abafaadbd78ab8711a P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.970536 24424 raft_consensus.cc:385] T a48e9f780b4e49abafaadbd78ab8711a P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:30.970638 24424 raft_consensus.cc:740] T a48e9f780b4e49abafaadbd78ab8711a P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50d44445234b4931aaa28141cdaa67ab, State: Initialized, Role: FOLLOWER
I20260810 01:57:30.970878 24424 consensus_queue.cc:260] T a48e9f780b4e49abafaadbd78ab8711a P 50d44445234b4931aaa28141cdaa67ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.971364 24424 ts_tablet_manager.cc:1434] T a48e9f780b4e49abafaadbd78ab8711a P 50d44445234b4931aaa28141cdaa67ab: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260810 01:57:30.975528 24421 tablet_bootstrap.cc:492] T a48e9f780b4e49abafaadbd78ab8711a P 48e33bc1e8e94d4887bb64214d0b6c36: Bootstrap starting.
I20260810 01:57:30.978885 24422 tablet_bootstrap.cc:492] T a48e9f780b4e49abafaadbd78ab8711a P 498d352dbea9405199a121e84a4ab926: Bootstrap starting.
I20260810 01:57:30.979254 24421 tablet_bootstrap.cc:654] T a48e9f780b4e49abafaadbd78ab8711a P 48e33bc1e8e94d4887bb64214d0b6c36: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:30.980682 24422 tablet_bootstrap.cc:654] T a48e9f780b4e49abafaadbd78ab8711a P 498d352dbea9405199a121e84a4ab926: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:30.985692 24422 tablet_bootstrap.cc:492] T a48e9f780b4e49abafaadbd78ab8711a P 498d352dbea9405199a121e84a4ab926: No bootstrap required, opened a new log
I20260810 01:57:30.985926 24422 ts_tablet_manager.cc:1403] T a48e9f780b4e49abafaadbd78ab8711a P 498d352dbea9405199a121e84a4ab926: Time spent bootstrapping tablet: real 0.007s user 0.007s sys 0.000s
I20260810 01:57:30.986788 24422 raft_consensus.cc:359] T a48e9f780b4e49abafaadbd78ab8711a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.987102 24422 raft_consensus.cc:385] T a48e9f780b4e49abafaadbd78ab8711a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:30.987233 24422 raft_consensus.cc:740] T a48e9f780b4e49abafaadbd78ab8711a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 498d352dbea9405199a121e84a4ab926, State: Initialized, Role: FOLLOWER
I20260810 01:57:30.987577 24422 consensus_queue.cc:260] T a48e9f780b4e49abafaadbd78ab8711a P 498d352dbea9405199a121e84a4ab926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.987718 24421 tablet_bootstrap.cc:492] T a48e9f780b4e49abafaadbd78ab8711a P 48e33bc1e8e94d4887bb64214d0b6c36: No bootstrap required, opened a new log
I20260810 01:57:30.987872 24421 ts_tablet_manager.cc:1403] T a48e9f780b4e49abafaadbd78ab8711a P 48e33bc1e8e94d4887bb64214d0b6c36: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.007s
I20260810 01:57:30.988641 24421 raft_consensus.cc:359] T a48e9f780b4e49abafaadbd78ab8711a P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.988888 24421 raft_consensus.cc:385] T a48e9f780b4e49abafaadbd78ab8711a P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:30.988991 24421 raft_consensus.cc:740] T a48e9f780b4e49abafaadbd78ab8711a P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48e33bc1e8e94d4887bb64214d0b6c36, State: Initialized, Role: FOLLOWER
I20260810 01:57:30.989346 24421 consensus_queue.cc:260] T a48e9f780b4e49abafaadbd78ab8711a P 48e33bc1e8e94d4887bb64214d0b6c36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:30.990833 24421 ts_tablet_manager.cc:1434] T a48e9f780b4e49abafaadbd78ab8711a P 48e33bc1e8e94d4887bb64214d0b6c36: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260810 01:57:30.994020 24422 ts_tablet_manager.cc:1434] T a48e9f780b4e49abafaadbd78ab8711a P 498d352dbea9405199a121e84a4ab926: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260810 01:57:31.013608 23389 catalog_manager.cc:2200] created HMS catalog entry for table [id=630b03060b7646118927a18573c0920c]
I20260810 01:57:31.020810 23389 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:45748 (request call id 5) took 1279 ms. Trace:
I20260810 01:57:31.020957 23389 rpcz_store.cc:276] 0810 01:57:29.741364 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 01:57:29.741422 (+ 58us) service_pool.cc:224] Handling call
0810 01:57:29.777955 (+ 36533us) catalog_manager.cc:2127] Acquired catalog manager lock
0810 01:57:29.778409 (+ 454us) catalog_manager.cc:2173] Created new table and tablet info
0810 01:57:31.013535 (+1235126us) catalog_manager.cc:2199] Created new table in HMS catalog
0810 01:57:31.020531 (+ 6996us) catalog_manager.cc:2233] Wrote table and tablets to system table
0810 01:57:31.020647 (+ 116us) catalog_manager.cc:2267] Inserted table and tablets into CatalogManager maps
0810 01:57:31.020763 (+ 116us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 01:57:31.014668 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 01:57:31.014815 (+ 147us) write_op.cc:432] Acquiring schema lock in shared mode
0810 01:57:31.014822 (+ 7us) write_op.cc:435] Acquired schema lock
0810 01:57:31.014825 (+ 3us) tablet.cc:662] Decoding operations
0810 01:57:31.014947 (+ 122us) write_op.cc:620] Acquiring the partition lock for write op
0810 01:57:31.014962 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0810 01:57:31.014967 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0810 01:57:31.015111 (+ 144us) tablet.cc:701] Row locks acquired
0810 01:57:31.015114 (+ 3us) write_op.cc:260] PREPARE: finished
0810 01:57:31.015163 (+ 49us) write_op.cc:270] Start()
0810 01:57:31.015204 (+ 41us) write_op.cc:276] Timestamp: P: 1786327051015094 usec, L: 0
0810 01:57:31.015208 (+ 4us) op_driver.cc:348] REPLICATION: starting
0810 01:57:31.015416 (+ 208us) log.cc:844] Serialized 432 byte log entry
0810 01:57:31.019173 (+ 3757us) op_driver.cc:464] REPLICATION: finished
0810 01:57:31.019330 (+ 157us) write_op.cc:301] APPLY: starting
0810 01:57:31.019345 (+ 15us) tablet.cc:1367] starting BulkCheckPresence
0810 01:57:31.019392 (+ 47us) tablet.cc:1370] finished BulkCheckPresence
0810 01:57:31.019416 (+ 24us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 01:57:31.019518 (+ 102us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 01:57:31.019553 (+ 35us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0810 01:57:31.019560 (+ 7us) write_op.cc:312] APPLY: finished
0810 01:57:31.019636 (+ 76us) log.cc:844] Serialized 28 byte log entry
0810 01:57:31.019702 (+ 66us) write_op.cc:489] Releasing partition, row and schema locks
0810 01:57:31.019723 (+ 21us) write_op.cc:454] Released schema lock
0810 01:57:31.019729 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0810 01:57:31.019806 (+ 77us) write_op.cc:489] Releasing partition, row and schema locks
0810 01:57:31.019811 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":1163578,"Hive Metastore.run_cpu_time_us":361,"Hive Metastore.run_wall_time_us":71367,"child_traces":[["op",{"apply.queue_time_us":90,"apply.run_cpu_time_us":464,"apply.run_wall_time_us":464,"num_ops":2,"prepare.queue_time_us":124,"prepare.run_cpu_time_us":887,"prepare.run_wall_time_us":887,"raft.queue_time_us":1176,"raft.run_cpu_time_us":328,"raft.run_wall_time_us":329,"replication_time_us":3937}]]}
F20260810 01:57:31.024713 23647 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 10 received events: NotificationEvent(eventId=1, eventTime=1786327049, eventType=CREATE_TABLE, dbName=default, tableName=table_4fb63c687c11493ca7c0b76735f137e9, message={"server":"","servicePrincipal":"","db":"default","table":"table_4fb63c687c11493ca7c0b76735f137e9","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_4fb63c687c11493ca7c0b76735f137e9\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1786327049},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/warehouse/table_4fb63c687c11493ca7c0b76735f137e9\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_4fb63c687c11493ca7c0b76735f137e9\",\"kudu.table_id\":\"cb924727e6014a2a9dd271f962df3807\",\"transient_lastDdlTime\":\"1786327049\",\"comment\":\"\",\"kudu.master_addresses\":\"127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"dfa48c56b63b4d0cbf3d6124921e8a17\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1786327049,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1786327049, eventType=CREATE_TABLE, dbName=default, tableName=table_c8ad906be0f74210912a57adead21255, message={"server":"","servicePrincipal":"","db":"default","table":"table_c8ad906be0f74210912a57adead21255","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_c8ad906be0f74210912a57adead21255\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1786327049},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/warehouse/table_c8ad906be0f74210912a57adead21255\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_c8ad906be0f74210912a57adead21255\",\"kudu.table_id\":\"4da3c920ea6a48588c9f0c91da00fb69\",\"transient_lastDdlTime\":\"1786327049\",\"comment\":\"\",\"kudu.master_addresses\":\"127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"dfa48c56b63b4d0cbf3d6124921e8a17\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1786327049,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
@ 0x7fe9df94fd5d google::LogMessage::Fail() at ??:0
@ 0x7fe9df953b23 google::LogMessage::SendToLog() at ??:0
@ 0x7fe9df94f75c google::LogMessage::Flush() at ??:0
@ 0x7fe9df94f9a1 google::LogMessage::~LogMessage() at ??:0
I20260810 01:57:31.042835 23746 tablet_service.cc:1511] Processing CreateTablet for tablet 2230d919953e49208659450a024c6a62 (DEFAULT_TABLE table=default.table_0ba2046f969346f68f1bde7cf6c9f273 [id=630b03060b7646118927a18573c0920c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:31.043326 23746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2230d919953e49208659450a024c6a62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:31.043641 24437 raft_consensus.cc:493] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:57:31.043777 24187 tablet_service.cc:1511] Processing CreateTablet for tablet 2230d919953e49208659450a024c6a62 (DEFAULT_TABLE table=default.table_0ba2046f969346f68f1bde7cf6c9f273 [id=630b03060b7646118927a18573c0920c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:31.043887 24437 raft_consensus.cc:515] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:31.044199 24187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2230d919953e49208659450a024c6a62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:31.044757 24437 leader_election.cc:290] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
I20260810 01:57:31.047180 23389 master_service.cc:960] ReplaceTablet: received request to replace tablet b24af70442e44ca4800206b1b5bd7715 from {username='slave'} at 127.0.0.1:45748
I20260810 01:57:31.047602 23964 tablet_service.cc:1511] Processing CreateTablet for tablet 2230d919953e49208659450a024c6a62 (DEFAULT_TABLE table=default.table_0ba2046f969346f68f1bde7cf6c9f273 [id=630b03060b7646118927a18573c0920c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:31.047930 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16db112abfb34d4a9f2ecbee98c39d8b" candidate_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50d44445234b4931aaa28141cdaa67ab" is_pre_election: true
I20260810 01:57:31.048020 23964 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2230d919953e49208659450a024c6a62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:31.047961 23389 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45748:
table { table_name: "default.Table_510ae1b5fcc74db788ffb64925cad250" } modify_external_catalogs: true
I20260810 01:57:31.048178 23776 raft_consensus.cc:2468] T 16db112abfb34d4a9f2ecbee98c39d8b P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48e33bc1e8e94d4887bb64214d0b6c36 in term 0.
I20260810 01:57:31.048208 23389 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45748:
table { table_name: "default.Table_510ae1b5fcc74db788ffb64925cad250" } modify_external_catalogs: true
I20260810 01:57:31.048676 23889 leader_election.cc:304] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab; no voters:
I20260810 01:57:31.048985 24437 raft_consensus.cc:2804] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 01:57:31.049081 24437 raft_consensus.cc:493] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 01:57:31.049182 24437 raft_consensus.cc:3060] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:31.051165 24222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16db112abfb34d4a9f2ecbee98c39d8b" candidate_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "498d352dbea9405199a121e84a4ab926" is_pre_election: true
I20260810 01:57:31.051362 24222 raft_consensus.cc:2468] T 16db112abfb34d4a9f2ecbee98c39d8b P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48e33bc1e8e94d4887bb64214d0b6c36 in term 0.
I20260810 01:57:31.053623 24437 raft_consensus.cc:515] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:31.054649 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16db112abfb34d4a9f2ecbee98c39d8b" candidate_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50d44445234b4931aaa28141cdaa67ab"
I20260810 01:57:31.054843 23776 raft_consensus.cc:3060] T 16db112abfb34d4a9f2ecbee98c39d8b P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:31.056496 24437 leader_election.cc:290] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 1 election: Requested vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
I20260810 01:57:31.056559 24222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16db112abfb34d4a9f2ecbee98c39d8b" candidate_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "498d352dbea9405199a121e84a4ab926"
I20260810 01:57:31.056725 24222 raft_consensus.cc:3060] T 16db112abfb34d4a9f2ecbee98c39d8b P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:31.059156 24421 tablet_bootstrap.cc:492] T 2230d919953e49208659450a024c6a62 P 48e33bc1e8e94d4887bb64214d0b6c36: Bootstrap starting.
I20260810 01:57:31.059697 24222 raft_consensus.cc:2468] T 16db112abfb34d4a9f2ecbee98c39d8b P 498d352dbea9405199a121e84a4ab926 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48e33bc1e8e94d4887bb64214d0b6c36 in term 1.
I20260810 01:57:31.060146 23892 leader_election.cc:304] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36, 498d352dbea9405199a121e84a4ab926; no voters:
I20260810 01:57:31.060494 24437 raft_consensus.cc:2804] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:57:31.060643 23776 raft_consensus.cc:2468] T 16db112abfb34d4a9f2ecbee98c39d8b P 50d44445234b4931aaa28141cdaa67ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48e33bc1e8e94d4887bb64214d0b6c36 in term 1.
I20260810 01:57:31.060674 24437 raft_consensus.cc:697] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 LEADER]: Becoming Leader. State: Replica: 48e33bc1e8e94d4887bb64214d0b6c36, State: Running, Role: LEADER
I20260810 01:57:31.060949 24437 consensus_queue.cc:237] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:31.061087 24421 tablet_bootstrap.cc:654] T 2230d919953e49208659450a024c6a62 P 48e33bc1e8e94d4887bb64214d0b6c36: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:31.062259 24424 tablet_bootstrap.cc:492] T 2230d919953e49208659450a024c6a62 P 50d44445234b4931aaa28141cdaa67ab: Bootstrap starting.
I20260810 01:57:31.063966 24424 tablet_bootstrap.cc:654] T 2230d919953e49208659450a024c6a62 P 50d44445234b4931aaa28141cdaa67ab: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:31.064111 24422 tablet_bootstrap.cc:492] T 2230d919953e49208659450a024c6a62 P 498d352dbea9405199a121e84a4ab926: Bootstrap starting.
@ 0x7fe9f6618f64 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260810 01:57:31.066119 24422 tablet_bootstrap.cc:654] T 2230d919953e49208659450a024c6a62 P 498d352dbea9405199a121e84a4ab926: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:31.068987 24421 tablet_bootstrap.cc:492] T 2230d919953e49208659450a024c6a62 P 48e33bc1e8e94d4887bb64214d0b6c36: No bootstrap required, opened a new log
I20260810 01:57:31.069142 24421 ts_tablet_manager.cc:1403] T 2230d919953e49208659450a024c6a62 P 48e33bc1e8e94d4887bb64214d0b6c36: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260810 01:57:31.070063 24421 raft_consensus.cc:359] T 2230d919953e49208659450a024c6a62 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:31.070258 24421 raft_consensus.cc:385] T 2230d919953e49208659450a024c6a62 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:31.070353 24421 raft_consensus.cc:740] T 2230d919953e49208659450a024c6a62 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48e33bc1e8e94d4887bb64214d0b6c36, State: Initialized, Role: FOLLOWER
I20260810 01:57:31.070616 24421 consensus_queue.cc:260] T 2230d919953e49208659450a024c6a62 P 48e33bc1e8e94d4887bb64214d0b6c36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:31.071331 24421 ts_tablet_manager.cc:1434] T 2230d919953e49208659450a024c6a62 P 48e33bc1e8e94d4887bb64214d0b6c36: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
@ 0x7fe9f6617811 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260810 01:57:31.078275 24424 tablet_bootstrap.cc:492] T 2230d919953e49208659450a024c6a62 P 50d44445234b4931aaa28141cdaa67ab: No bootstrap required, opened a new log
@ 0x7fe9f661f48c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260810 01:57:31.078431 24424 ts_tablet_manager.cc:1403] T 2230d919953e49208659450a024c6a62 P 50d44445234b4931aaa28141cdaa67ab: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.010s
I20260810 01:57:31.079380 24424 raft_consensus.cc:359] T 2230d919953e49208659450a024c6a62 P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:31.079704 24424 raft_consensus.cc:385] T 2230d919953e49208659450a024c6a62 P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:31.079818 24424 raft_consensus.cc:740] T 2230d919953e49208659450a024c6a62 P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50d44445234b4931aaa28141cdaa67ab, State: Initialized, Role: FOLLOWER
I20260810 01:57:31.080118 24424 consensus_queue.cc:260] T 2230d919953e49208659450a024c6a62 P 50d44445234b4931aaa28141cdaa67ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:31.086676 24424 ts_tablet_manager.cc:1434] T 2230d919953e49208659450a024c6a62 P 50d44445234b4931aaa28141cdaa67ab: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260810 01:57:31.091854 24422 tablet_bootstrap.cc:492] T 2230d919953e49208659450a024c6a62 P 498d352dbea9405199a121e84a4ab926: No bootstrap required, opened a new log
I20260810 01:57:31.092023 24422 ts_tablet_manager.cc:1403] T 2230d919953e49208659450a024c6a62 P 498d352dbea9405199a121e84a4ab926: Time spent bootstrapping tablet: real 0.028s user 0.009s sys 0.004s
@ 0x7fe9f661f309 std::_Function_handler<>::_M_invoke() at ??:0
I20260810 01:57:31.092839 24422 raft_consensus.cc:359] T 2230d919953e49208659450a024c6a62 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:31.095319 24422 raft_consensus.cc:385] T 2230d919953e49208659450a024c6a62 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:31.095434 24422 raft_consensus.cc:740] T 2230d919953e49208659450a024c6a62 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 498d352dbea9405199a121e84a4ab926, State: Initialized, Role: FOLLOWER
I20260810 01:57:31.095731 24422 consensus_queue.cc:260] T 2230d919953e49208659450a024c6a62 P 498d352dbea9405199a121e84a4ab926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
@ 0xaaf86d std::function<>::operator()() at ??:?
I20260810 01:57:31.102020 24422 ts_tablet_manager.cc:1434] T 2230d919953e49208659450a024c6a62 P 498d352dbea9405199a121e84a4ab926: Time spent starting tablet: real 0.010s user 0.001s sys 0.001s
@ 0x7fe9e2677dd9 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fe9e5d476db start_thread at ??:0
@ 0x7fe9de7fe61f clone at ??:0
I20260810 01:57:31.114147 24438 raft_consensus.cc:493] T 4579a90be0b6409781fd9e470324192c P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:57:31.114326 24438 raft_consensus.cc:515] T 4579a90be0b6409781fd9e470324192c P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:31.115090 24438 leader_election.cc:290] T 4579a90be0b6409781fd9e470324192c P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
I20260810 01:57:31.116012 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4579a90be0b6409781fd9e470324192c" candidate_uuid: "498d352dbea9405199a121e84a4ab926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50d44445234b4931aaa28141cdaa67ab" is_pre_election: true
I20260810 01:57:31.116235 23776 raft_consensus.cc:2468] T 4579a90be0b6409781fd9e470324192c P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 498d352dbea9405199a121e84a4ab926 in term 0.
I20260810 01:57:31.116276 23998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4579a90be0b6409781fd9e470324192c" candidate_uuid: "498d352dbea9405199a121e84a4ab926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" is_pre_election: true
I20260810 01:57:31.116506 23998 raft_consensus.cc:2468] T 4579a90be0b6409781fd9e470324192c P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 498d352dbea9405199a121e84a4ab926 in term 0.
I20260810 01:57:31.116705 24109 leader_election.cc:304] T 4579a90be0b6409781fd9e470324192c P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab; no voters:
I20260810 01:57:31.117250 24438 raft_consensus.cc:2804] T 4579a90be0b6409781fd9e470324192c P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 01:57:31.117364 24438 raft_consensus.cc:493] T 4579a90be0b6409781fd9e470324192c P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 01:57:31.117477 24438 raft_consensus.cc:3060] T 4579a90be0b6409781fd9e470324192c P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:31.120987 24438 raft_consensus.cc:515] T 4579a90be0b6409781fd9e470324192c P 498d352dbea9405199a121e84a4ab926 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:31.122025 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4579a90be0b6409781fd9e470324192c" candidate_uuid: "498d352dbea9405199a121e84a4ab926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50d44445234b4931aaa28141cdaa67ab"
I20260810 01:57:31.122211 23776 raft_consensus.cc:3060] T 4579a90be0b6409781fd9e470324192c P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:31.122262 24438 leader_election.cc:290] T 4579a90be0b6409781fd9e470324192c P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 election: Requested vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
I20260810 01:57:31.122591 23998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4579a90be0b6409781fd9e470324192c" candidate_uuid: "498d352dbea9405199a121e84a4ab926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48e33bc1e8e94d4887bb64214d0b6c36"
I20260810 01:57:31.122759 23998 raft_consensus.cc:3060] T 4579a90be0b6409781fd9e470324192c P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:31.125432 23776 raft_consensus.cc:2468] T 4579a90be0b6409781fd9e470324192c P 50d44445234b4931aaa28141cdaa67ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 498d352dbea9405199a121e84a4ab926 in term 1.
I20260810 01:57:31.125778 23998 raft_consensus.cc:2468] T 4579a90be0b6409781fd9e470324192c P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 498d352dbea9405199a121e84a4ab926 in term 1.
I20260810 01:57:31.126295 24109 leader_election.cc:304] T 4579a90be0b6409781fd9e470324192c P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab; no voters:
I20260810 01:57:31.126844 24438 raft_consensus.cc:2804] T 4579a90be0b6409781fd9e470324192c P 498d352dbea9405199a121e84a4ab926 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:57:31.127117 24438 raft_consensus.cc:697] T 4579a90be0b6409781fd9e470324192c P 498d352dbea9405199a121e84a4ab926 [term 1 LEADER]: Becoming Leader. State: Replica: 498d352dbea9405199a121e84a4ab926, State: Running, Role: LEADER
I20260810 01:57:31.127485 24438 consensus_queue.cc:237] T 4579a90be0b6409781fd9e470324192c P 498d352dbea9405199a121e84a4ab926 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:31.291221 24437 raft_consensus.cc:493] T a48e9f780b4e49abafaadbd78ab8711a P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:57:31.291460 24437 raft_consensus.cc:515] T a48e9f780b4e49abafaadbd78ab8711a P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:31.292297 24437 leader_election.cc:290] T a48e9f780b4e49abafaadbd78ab8711a P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
I20260810 01:57:31.292909 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48e9f780b4e49abafaadbd78ab8711a" candidate_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50d44445234b4931aaa28141cdaa67ab" is_pre_election: true
I20260810 01:57:31.293061 24222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48e9f780b4e49abafaadbd78ab8711a" candidate_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "498d352dbea9405199a121e84a4ab926" is_pre_election: true
I20260810 01:57:31.293160 23776 raft_consensus.cc:2468] T a48e9f780b4e49abafaadbd78ab8711a P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48e33bc1e8e94d4887bb64214d0b6c36 in term 0.
I20260810 01:57:31.293319 24222 raft_consensus.cc:2468] T a48e9f780b4e49abafaadbd78ab8711a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48e33bc1e8e94d4887bb64214d0b6c36 in term 0.
I20260810 01:57:31.293707 23889 leader_election.cc:304] T a48e9f780b4e49abafaadbd78ab8711a P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab; no voters:
I20260810 01:57:31.294004 24437 raft_consensus.cc:2804] T a48e9f780b4e49abafaadbd78ab8711a P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 01:57:31.294131 24437 raft_consensus.cc:493] T a48e9f780b4e49abafaadbd78ab8711a P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 01:57:31.294248 24437 raft_consensus.cc:3060] T a48e9f780b4e49abafaadbd78ab8711a P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:31.297641 24437 raft_consensus.cc:515] T a48e9f780b4e49abafaadbd78ab8711a P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:31.298681 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48e9f780b4e49abafaadbd78ab8711a" candidate_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50d44445234b4931aaa28141cdaa67ab"
I20260810 01:57:31.298751 24222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48e9f780b4e49abafaadbd78ab8711a" candidate_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "498d352dbea9405199a121e84a4ab926"
I20260810 01:57:31.298889 23776 raft_consensus.cc:3060] T a48e9f780b4e49abafaadbd78ab8711a P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:31.298974 24222 raft_consensus.cc:3060] T a48e9f780b4e49abafaadbd78ab8711a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:31.302167 24222 raft_consensus.cc:2468] T a48e9f780b4e49abafaadbd78ab8711a P 498d352dbea9405199a121e84a4ab926 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48e33bc1e8e94d4887bb64214d0b6c36 in term 1.
I20260810 01:57:31.302513 23776 raft_consensus.cc:2468] T a48e9f780b4e49abafaadbd78ab8711a P 50d44445234b4931aaa28141cdaa67ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48e33bc1e8e94d4887bb64214d0b6c36 in term 1.
I20260810 01:57:31.302597 23892 leader_election.cc:304] T a48e9f780b4e49abafaadbd78ab8711a P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36, 498d352dbea9405199a121e84a4ab926; no voters:
I20260810 01:57:31.303016 24490 raft_consensus.cc:2804] T a48e9f780b4e49abafaadbd78ab8711a P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:57:31.303588 24490 raft_consensus.cc:697] T a48e9f780b4e49abafaadbd78ab8711a P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 LEADER]: Becoming Leader. State: Replica: 48e33bc1e8e94d4887bb64214d0b6c36, State: Running, Role: LEADER
I20260810 01:57:31.303933 24490 consensus_queue.cc:237] T a48e9f780b4e49abafaadbd78ab8711a P 48e33bc1e8e94d4887bb64214d0b6c36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:31.302997 24437 leader_election.cc:290] T a48e9f780b4e49abafaadbd78ab8711a P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 1 election: Requested vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 01:57:31.309504 24467 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:57:31.309826 24467 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:57:31.309983 24467 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260810 01:57:31.325093 24467 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:57:31.325233 24467 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:57:31.325376 24467 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:57:31.325471 24467 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:57:31.337431 24467 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.69.20:43209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:37513
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.69.60:36917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.69.60
--webserver_port=44467
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 10 Aug 2026 01:44:04 UTC on e7f111948823
build id 13266
ASAN enabled
I20260810 01:57:31.339432 24467 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:57:31.341934 24467 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260810 01:57:31.351950 24548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:57:31.354339 24554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:57:31.355762 24317 connection.cc:573] client connection to 127.22.69.61:41945 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 01:57:31.356715 24317 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.22.69.61:41945)
W20260810 01:57:31.356978 24317 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.22.69.61:41945)
W20260810 01:57:31.357482 24317 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.22.69.61:41945)
W20260810 01:57:31.358814 24317 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.22.69.61:41945)
W20260810 01:57:31.367643 24444 negotiation.cc:336] Failed RPC negotiation. Trace:
0810 01:57:31.363242 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.22.69.61:41945 (local address 127.22.69.2:48711)
0810 01:57:31.363533 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0810 01:57:31.363553 (+ 20us) client_negotiation.cc:175] Beginning negotiation
0810 01:57:31.363690 (+ 137us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0810 01:57:31.366756 (+ 3066us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":168}
W20260810 01:57:31.368250 24557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:57:31.369445 24487 negotiation.cc:336] Failed RPC negotiation. Trace:
0810 01:57:31.364128 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.22.69.61:41945 (local address 127.22.69.3:50283)
0810 01:57:31.364410 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0810 01:57:31.364429 (+ 19us) client_negotiation.cc:175] Beginning negotiation
0810 01:57:31.364551 (+ 122us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0810 01:57:31.368706 (+ 4155us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":175}
W20260810 01:57:31.374284 24559 negotiation.cc:336] Failed RPC negotiation. Trace:
0810 01:57:31.359928 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.22.69.61:41945 (local address 127.0.0.1:45754)
0810 01:57:31.373837 (+ 13909us) negotiation.cc:107] Waiting for socket to connect
0810 01:57:31.374016 (+ 179us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":13755,"threads_started":1}
I20260810 01:57:31.375025 24438 raft_consensus.cc:493] T 89d9556cc526444299c786de606bb5b1 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:57:31.375265 24438 raft_consensus.cc:515] T 89d9556cc526444299c786de606bb5b1 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:31.376039 24438 leader_election.cc:290] T 89d9556cc526444299c786de606bb5b1 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
I20260810 01:57:31.376618 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89d9556cc526444299c786de606bb5b1" candidate_uuid: "498d352dbea9405199a121e84a4ab926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50d44445234b4931aaa28141cdaa67ab" is_pre_election: true
I20260810 01:57:31.376721 23998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89d9556cc526444299c786de606bb5b1" candidate_uuid: "498d352dbea9405199a121e84a4ab926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" is_pre_election: true
I20260810 01:57:31.376854 23776 raft_consensus.cc:2468] T 89d9556cc526444299c786de606bb5b1 P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 498d352dbea9405199a121e84a4ab926 in term 0.
I20260810 01:57:31.377027 23998 raft_consensus.cc:2468] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 498d352dbea9405199a121e84a4ab926 in term 0.
I20260810 01:57:31.377353 24467 server_base.cc:1061] running on GCE node
I20260810 01:57:31.377388 24109 leader_election.cc:304] T 89d9556cc526444299c786de606bb5b1 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab; no voters:
I20260810 01:57:31.377710 24438 raft_consensus.cc:2804] T 89d9556cc526444299c786de606bb5b1 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 01:57:31.377841 24438 raft_consensus.cc:493] T 89d9556cc526444299c786de606bb5b1 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 01:57:31.378000 24438 raft_consensus.cc:3060] T 89d9556cc526444299c786de606bb5b1 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:31.378575 24467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:57:31.383533 24438 raft_consensus.cc:515] T 89d9556cc526444299c786de606bb5b1 P 498d352dbea9405199a121e84a4ab926 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:31.384948 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89d9556cc526444299c786de606bb5b1" candidate_uuid: "498d352dbea9405199a121e84a4ab926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50d44445234b4931aaa28141cdaa67ab"
I20260810 01:57:31.385241 23776 raft_consensus.cc:3060] T 89d9556cc526444299c786de606bb5b1 P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:31.389710 23776 raft_consensus.cc:2468] T 89d9556cc526444299c786de606bb5b1 P 50d44445234b4931aaa28141cdaa67ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 498d352dbea9405199a121e84a4ab926 in term 1.
I20260810 01:57:31.390203 24467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:57:31.390378 24109 leader_election.cc:304] T 89d9556cc526444299c786de606bb5b1 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab; no voters:
I20260810 01:57:31.390730 24493 raft_consensus.cc:2804] T 89d9556cc526444299c786de606bb5b1 P 498d352dbea9405199a121e84a4ab926 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:57:31.390853 24438 leader_election.cc:290] T 89d9556cc526444299c786de606bb5b1 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 election: Requested vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
I20260810 01:57:31.392683 24493 raft_consensus.cc:697] T 89d9556cc526444299c786de606bb5b1 P 498d352dbea9405199a121e84a4ab926 [term 1 LEADER]: Becoming Leader. State: Replica: 498d352dbea9405199a121e84a4ab926, State: Running, Role: LEADER
I20260810 01:57:31.393046 24493 consensus_queue.cc:237] T 89d9556cc526444299c786de606bb5b1 P 498d352dbea9405199a121e84a4ab926 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:31.394667 23998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89d9556cc526444299c786de606bb5b1" candidate_uuid: "498d352dbea9405199a121e84a4ab926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48e33bc1e8e94d4887bb64214d0b6c36"
I20260810 01:57:31.394979 23998 raft_consensus.cc:3060] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:31.402935 23998 raft_consensus.cc:2468] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 498d352dbea9405199a121e84a4ab926 in term 1.
I20260810 01:57:31.403334 24467 hybrid_clock.cc:648] HybridClock initialized: now 1786327051402505 us; error 812 us; skew 500 ppm
I20260810 01:57:31.403892 24467 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 01:57:31.407032 24467 webserver.cc:533] Webserver started at http://127.22.69.60:44467/ using document root <none> and password file <none>
I20260810 01:57:31.408074 24467 fs_manager.cc:362] Metadata directory not provided
I20260810 01:57:31.408209 24467 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:57:31.414474 24467 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260810 01:57:31.418234 24571 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:57:31.419761 24467 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 01:57:31.419946 24467 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/data,/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/wal
uuid: "757bb3f336084510845ed58df264c721"
format_stamp: "Formatted at 2026-08-10 01:57:26 on dist-test-slave-n326"
I20260810 01:57:31.420660 24467 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:57:31.461918 24467 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260810 01:57:31.463681 24467 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:57:31.463985 24467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:57:31.467913 24430 consensus_queue.cc:1048] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 50d44445234b4931aaa28141cdaa67ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:57:31.486972 24602 consensus_queue.cc:1048] T 1c3db8429f9f4cf5b29d3d1a6cc0651c P 50d44445234b4931aaa28141cdaa67ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:57:31.511794 24493 consensus_queue.cc:1048] T e6ef3afe40a847dbb08792578d52401d P 498d352dbea9405199a121e84a4ab926 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:57:31.519860 24493 consensus_queue.cc:1048] T e6ef3afe40a847dbb08792578d52401d P 498d352dbea9405199a121e84a4ab926 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:57:31.531503 24437 consensus_queue.cc:1048] T 1844a63489d74f1c919e96501e4bd568 P 48e33bc1e8e94d4887bb64214d0b6c36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:57:31.534700 24493 raft_consensus.cc:493] T 2230d919953e49208659450a024c6a62 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:57:31.534912 24493 raft_consensus.cc:515] T 2230d919953e49208659450a024c6a62 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:31.535668 24493 leader_election.cc:290] T 2230d919953e49208659450a024c6a62 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
I20260810 01:57:31.536650 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2230d919953e49208659450a024c6a62" candidate_uuid: "498d352dbea9405199a121e84a4ab926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50d44445234b4931aaa28141cdaa67ab" is_pre_election: true
I20260810 01:57:31.536863 23998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2230d919953e49208659450a024c6a62" candidate_uuid: "498d352dbea9405199a121e84a4ab926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" is_pre_election: true
I20260810 01:57:31.536948 23776 raft_consensus.cc:2468] T 2230d919953e49208659450a024c6a62 P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 498d352dbea9405199a121e84a4ab926 in term 0.
I20260810 01:57:31.537115 23998 raft_consensus.cc:2468] T 2230d919953e49208659450a024c6a62 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 498d352dbea9405199a121e84a4ab926 in term 0.
I20260810 01:57:31.537715 24109 leader_election.cc:304] T 2230d919953e49208659450a024c6a62 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab; no voters:
I20260810 01:57:31.538431 24493 raft_consensus.cc:2804] T 2230d919953e49208659450a024c6a62 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 01:57:31.538540 24493 raft_consensus.cc:493] T 2230d919953e49208659450a024c6a62 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 01:57:31.538657 24493 raft_consensus.cc:3060] T 2230d919953e49208659450a024c6a62 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:31.539861 24544 consensus_queue.cc:1048] T 1844a63489d74f1c919e96501e4bd568 P 48e33bc1e8e94d4887bb64214d0b6c36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:57:31.541982 24493 raft_consensus.cc:515] T 2230d919953e49208659450a024c6a62 P 498d352dbea9405199a121e84a4ab926 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:31.546936 23998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2230d919953e49208659450a024c6a62" candidate_uuid: "498d352dbea9405199a121e84a4ab926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48e33bc1e8e94d4887bb64214d0b6c36"
I20260810 01:57:31.547160 23998 raft_consensus.cc:3060] T 2230d919953e49208659450a024c6a62 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:31.547817 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2230d919953e49208659450a024c6a62" candidate_uuid: "498d352dbea9405199a121e84a4ab926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50d44445234b4931aaa28141cdaa67ab"
I20260810 01:57:31.548032 23776 raft_consensus.cc:3060] T 2230d919953e49208659450a024c6a62 P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:31.550472 23998 raft_consensus.cc:2468] T 2230d919953e49208659450a024c6a62 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 498d352dbea9405199a121e84a4ab926 in term 1.
I20260810 01:57:31.551105 24109 leader_election.cc:304] T 2230d919953e49208659450a024c6a62 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36, 498d352dbea9405199a121e84a4ab926; no voters:
I20260810 01:57:31.551528 24438 raft_consensus.cc:2804] T 2230d919953e49208659450a024c6a62 P 498d352dbea9405199a121e84a4ab926 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:57:31.551723 24438 raft_consensus.cc:697] T 2230d919953e49208659450a024c6a62 P 498d352dbea9405199a121e84a4ab926 [term 1 LEADER]: Becoming Leader. State: Replica: 498d352dbea9405199a121e84a4ab926, State: Running, Role: LEADER
I20260810 01:57:31.552100 24438 consensus_queue.cc:237] T 2230d919953e49208659450a024c6a62 P 498d352dbea9405199a121e84a4ab926 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:31.552598 23776 raft_consensus.cc:2468] T 2230d919953e49208659450a024c6a62 P 50d44445234b4931aaa28141cdaa67ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 498d352dbea9405199a121e84a4ab926 in term 1.
I20260810 01:57:31.558609 24493 leader_election.cc:290] T 2230d919953e49208659450a024c6a62 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 election: Requested vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
I20260810 01:57:31.559490 24657 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 01:57:31.559720 24657 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:31.567310 24657 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:57:31.568898 23260 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111) [suppressed 7 similar messages]
W20260810 01:57:31.577417 23260 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
W20260810 01:57:31.577965 23260 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
I20260810 01:57:31.578128 23260 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 01:57:31.578545 24657 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260810 01:57:31.582736 24467 rpc_server.cc:307] RPC server started. Bound to: 127.22.69.60:36917
I20260810 01:57:31.582768 24689 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.22.69.60:36917 every 8 connection(s)
I20260810 01:57:31.584434 24467 server_base.cc:1193] Dumped server information to /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/data/info.pb
I20260810 01:57:31.587376 22804 external_mini_cluster.cc:1434] Started /tmp/dist-test-task_qhVVJ/build/asan/bin/kudu as pid 24467
I20260810 01:57:31.592989 24691 sys_catalog.cc:263] Verifying existing consensus state
I20260810 01:57:31.599169 24691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721: Bootstrap starting.
I20260810 01:57:31.609416 24691 log.cc:826] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721: Log is configured to *not* fsync() on all Append() calls
I20260810 01:57:31.617211 24691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 01:57:31.617784 24691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721: Bootstrap complete.
I20260810 01:57:31.623426 24691 raft_consensus.cc:359] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:31.624135 24691 raft_consensus.cc:740] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 757bb3f336084510845ed58df264c721, State: Initialized, Role: FOLLOWER
I20260810 01:57:31.624789 24691 consensus_queue.cc:260] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:31.625715 24697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:31.625973 24697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:57:31.626720 24691 sys_catalog.cc:565] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:57:31.640632 24720 catalog_manager.cc:1295] Loaded cluster ID: dfa48c56b63b4d0cbf3d6124921e8a17
I20260810 01:57:31.640748 24720 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721: loading cluster ID for follower catalog manager: success
I20260810 01:57:31.645953 24720 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721: acquiring CA information for follower catalog manager: success
I20260810 01:57:31.649096 24720 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 01:57:31.662281 24467 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 01:57:31.665992 24467 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 01:57:31.780730 24438 consensus_queue.cc:1048] T 09ed68395e2d44a5b51a22abe2203406 P 498d352dbea9405199a121e84a4ab926 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:57:31.788275 24493 consensus_queue.cc:1048] T 09ed68395e2d44a5b51a22abe2203406 P 498d352dbea9405199a121e84a4ab926 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:57:31.799726 24298 heartbeater.cc:344] Connected to a master server at 127.22.69.60:36917
I20260810 01:57:31.799932 24298 heartbeater.cc:499] Master 127.22.69.60:36917 was elected leader, sending a full tablet report...
I20260810 01:57:31.803604 24063 heartbeater.cc:344] Connected to a master server at 127.22.69.60:36917
I20260810 01:57:31.805404 23841 heartbeater.cc:344] Connected to a master server at 127.22.69.60:36917
I20260810 01:57:31.808862 24614 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" instance_seqno: 1786327048775850) as {username='slave'} at 127.22.69.2:52095; Asking this server to re-register.
I20260810 01:57:31.808893 24616 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "498d352dbea9405199a121e84a4ab926" instance_seqno: 1786327049319804) as {username='slave'} at 127.22.69.3:43215; Asking this server to re-register.
I20260810 01:57:31.808951 24610 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" instance_seqno: 1786327048176732) as {username='slave'} at 127.22.69.1:37161; Asking this server to re-register.
I20260810 01:57:31.810051 24298 heartbeater.cc:461] Registering TS with master...
I20260810 01:57:31.810820 24063 heartbeater.cc:461] Registering TS with master...
I20260810 01:57:31.812176 23841 heartbeater.cc:461] Registering TS with master...
I20260810 01:57:31.812217 24610 ts_manager.cc:194] Registered new tserver with Master: 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
I20260810 01:57:31.813956 24616 ts_manager.cc:194] Registered new tserver with Master: 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
I20260810 01:57:31.814174 24610 ts_manager.cc:194] Registered new tserver with Master: 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491)
I20260810 01:57:31.992082 24493 consensus_queue.cc:1048] T 4579a90be0b6409781fd9e470324192c P 498d352dbea9405199a121e84a4ab926 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:57:31.998121 24564 consensus_queue.cc:1048] T 4579a90be0b6409781fd9e470324192c P 498d352dbea9405199a121e84a4ab926 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:57:32.014696 24544 consensus_queue.cc:1048] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:57:32.020605 24490 consensus_queue.cc:1048] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:57:32.093497 23234 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 01:57:32.093691 23234 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 01:57:32.205518 24736 raft_consensus.cc:493] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:57:32.205842 24736 raft_consensus.cc:515] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:32.208263 24736 leader_election.cc:290] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), b1422349180c446db6c1768b3025253c (127.22.69.61:41945)
I20260810 01:57:32.208873 23301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "757bb3f336084510845ed58df264c721" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "919313c0dee540a690f46dbc8208d20b" is_pre_election: true
I20260810 01:57:32.209148 23301 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 757bb3f336084510845ed58df264c721 in current term 2: Already voted for candidate b1422349180c446db6c1768b3025253c in this term.
W20260810 01:57:32.209435 24574 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 01:57:32.210907 24574 leader_election.cc:336] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
I20260810 01:57:32.211046 24574 leader_election.cc:304] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 757bb3f336084510845ed58df264c721; no voters: 919313c0dee540a690f46dbc8208d20b, b1422349180c446db6c1768b3025253c
I20260810 01:57:32.211453 24736 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 1 FOLLOWER]: Advancing to term 2
I20260810 01:57:32.215276 24736 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 01:57:32.250439 24741 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 01:57:32.250661 24741 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:32.251381 24741 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:57:32.254179 23260 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
I20260810 01:57:32.257978 24631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "919313c0dee540a690f46dbc8208d20b" candidate_term: 3 candidate_status { last_received { term: 2 index: 32 } } ignore_live_leader: false dest_uuid: "757bb3f336084510845ed58df264c721" is_pre_election: true
I20260810 01:57:32.258363 24631 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 919313c0dee540a690f46dbc8208d20b in term 2.
I20260810 01:57:32.258800 23260 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 757bb3f336084510845ed58df264c721, 919313c0dee540a690f46dbc8208d20b; no voters: b1422349180c446db6c1768b3025253c
I20260810 01:57:32.259119 24741 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260810 01:57:32.259243 24741 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 2 FOLLOWER]: Starting leader election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 01:57:32.259359 24741 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 2 FOLLOWER]: Advancing to term 3
I20260810 01:57:32.261469 24741 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:32.261963 24741 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 3 election: Requested vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
I20260810 01:57:32.262708 24631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "919313c0dee540a690f46dbc8208d20b" candidate_term: 3 candidate_status { last_received { term: 2 index: 32 } } ignore_live_leader: false dest_uuid: "757bb3f336084510845ed58df264c721"
I20260810 01:57:32.262902 24631 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 2 FOLLOWER]: Advancing to term 3
W20260810 01:57:32.263401 23260 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
I20260810 01:57:32.265623 24631 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 919313c0dee540a690f46dbc8208d20b in term 3.
I20260810 01:57:32.266036 23260 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 757bb3f336084510845ed58df264c721, 919313c0dee540a690f46dbc8208d20b; no voters: b1422349180c446db6c1768b3025253c
I20260810 01:57:32.266382 24741 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 3 FOLLOWER]: Leader election won for term 3
I20260810 01:57:32.266901 24741 raft_consensus.cc:697] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 3 LEADER]: Becoming Leader. State: Replica: 919313c0dee540a690f46dbc8208d20b, State: Running, Role: LEADER
I20260810 01:57:32.267491 24741 consensus_queue.cc:237] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 2.32, Last appended by leader: 32, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:32.275601 24749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 919313c0dee540a690f46dbc8208d20b. Latest consensus state: current_term: 3 leader_uuid: "919313c0dee540a690f46dbc8208d20b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:32.275913 24749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: This master's current role is: LEADER
I20260810 01:57:32.276620 24753 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260810 01:57:32.279306 24753 catalog_manager.cc:679] Loaded metadata for table default.table_72b6f93a124042e4a090dc194134c75a [id=017756566b72415aa33842f8df2b3e91]
I20260810 01:57:32.279788 24753 catalog_manager.cc:679] Loaded metadata for table default.table_73d521a2c5504815ad6d7cc3dd0bf818 [id=2f1832807601444a88d61b8396a67db9]
I20260810 01:57:32.280176 24753 catalog_manager.cc:679] Loaded metadata for table default.table_c8ad906be0f74210912a57adead21255 [id=4da3c920ea6a48588c9f0c91da00fb69]
I20260810 01:57:32.280558 24753 catalog_manager.cc:679] Loaded metadata for table default.table_983bf2e604df4fdbac75c5aca292f736 [id=552771dde220474a9be9abeefd3a0472]
I20260810 01:57:32.280911 24753 catalog_manager.cc:679] Loaded metadata for table default.table_22726fd1a11f409f9f5287ab16d8e551 [id=5edf9cb336414ae3bd41e9f5ca41fcca]
I20260810 01:57:32.281283 24753 catalog_manager.cc:679] Loaded metadata for table default.table_0ba2046f969346f68f1bde7cf6c9f273 [id=630b03060b7646118927a18573c0920c]
I20260810 01:57:32.281652 24753 catalog_manager.cc:679] Loaded metadata for table default.table_510ae1b5fcc74db788ffb64925cad250 [id=7d240d10089c47eaa014d59c4ac7f0b4]
I20260810 01:57:32.282014 24753 catalog_manager.cc:679] Loaded metadata for table default.table_379ea52611ef49d591a022addb7afc0c [id=87ba140e9080402aa494198d6f212f05]
I20260810 01:57:32.282352 24753 catalog_manager.cc:679] Loaded metadata for table default.table_57697192c9bf44969c2b200e94865a3e [id=ef2e836fffc64aa6a8ddc0425f39e317]
I20260810 01:57:32.283995 24753 tablet_loader.cc:96] loaded metadata for tablet 09ed68395e2d44a5b51a22abe2203406 (table default.table_73d521a2c5504815ad6d7cc3dd0bf818 [id=2f1832807601444a88d61b8396a67db9])
I20260810 01:57:32.284564 24753 tablet_loader.cc:96] loaded metadata for tablet 16db112abfb34d4a9f2ecbee98c39d8b (table default.table_72b6f93a124042e4a090dc194134c75a [id=017756566b72415aa33842f8df2b3e91])
I20260810 01:57:32.284921 24753 tablet_loader.cc:96] loaded metadata for tablet 1844a63489d74f1c919e96501e4bd568 (table default.table_c8ad906be0f74210912a57adead21255 [id=4da3c920ea6a48588c9f0c91da00fb69])
I20260810 01:57:32.285251 24753 tablet_loader.cc:96] loaded metadata for tablet 1c3db8429f9f4cf5b29d3d1a6cc0651c (table default.table_379ea52611ef49d591a022addb7afc0c [id=87ba140e9080402aa494198d6f212f05])
I20260810 01:57:32.285583 24753 tablet_loader.cc:96] loaded metadata for tablet 2230d919953e49208659450a024c6a62 (table default.table_0ba2046f969346f68f1bde7cf6c9f273 [id=630b03060b7646118927a18573c0920c])
I20260810 01:57:32.285887 24753 tablet_loader.cc:96] loaded metadata for tablet 4579a90be0b6409781fd9e470324192c (table default.table_22726fd1a11f409f9f5287ab16d8e551 [id=5edf9cb336414ae3bd41e9f5ca41fcca])
I20260810 01:57:32.286206 24753 tablet_loader.cc:96] loaded metadata for tablet 89d9556cc526444299c786de606bb5b1 (table default.table_510ae1b5fcc74db788ffb64925cad250 [id=7d240d10089c47eaa014d59c4ac7f0b4])
I20260810 01:57:32.286516 24753 tablet_loader.cc:96] loaded metadata for tablet a48e9f780b4e49abafaadbd78ab8711a (table default.table_57697192c9bf44969c2b200e94865a3e [id=ef2e836fffc64aa6a8ddc0425f39e317])
I20260810 01:57:32.287200 24753 tablet_loader.cc:96] loaded metadata for tablet e6ef3afe40a847dbb08792578d52401d (table default.table_983bf2e604df4fdbac75c5aca292f736 [id=552771dde220474a9be9abeefd3a0472])
I20260810 01:57:32.287451 24753 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260810 01:57:32.288311 24753 catalog_manager.cc:1295] Loaded cluster ID: dfa48c56b63b4d0cbf3d6124921e8a17
I20260810 01:57:32.288412 24753 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260810 01:57:32.289634 24753 catalog_manager.cc:1540] Loading token signing keys...
I20260810 01:57:32.290512 24753 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b: Loaded TSK: 0
I20260810 01:57:32.291594 24753 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260810 01:57:32.292550 24753 catalog_manager.cc:1563] Loading latest processed Hive Metastore notification log event ID...
I20260810 01:57:32.293285 24753 catalog_manager.cc:5941] Last processed Hive Metastore notification event ID: -1
I20260810 01:57:32.322386 23284 master_service.cc:960] ReplaceTablet: received request to replace tablet 1844a63489d74f1c919e96501e4bd568 from {username='slave'} at 127.0.0.1:58158
I20260810 01:57:32.322968 23285 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58158:
name: "default.Table_e3c77b251fdf471d8916f06da94be990"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:32.324633 23279 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58158:
table { table_name: "default.Table_0ba2046f969346f68f1bde7cf6c9f273" } modify_external_catalogs: true
I20260810 01:57:32.323423 23283 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58158:
name: "default.Table_b03802d8f2f14a24b00ab6efbe9392dc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:32.327486 23279 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58158:
table { table_name: "default.Table_0ba2046f969346f68f1bde7cf6c9f273" } modify_external_catalogs: true
I20260810 01:57:32.325963 23278 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58158:
table { table_name: "default.Table_57697192c9bf44969c2b200e94865a3e" } modify_external_catalogs: true
I20260810 01:57:32.328274 23278 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58158:
table { table_name: "default.Table_57697192c9bf44969c2b200e94865a3e" } modify_external_catalogs: true
I20260810 01:57:32.325636 23276 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58158:
table { table_name: "default.Table_22726fd1a11f409f9f5287ab16d8e551" } modify_external_catalogs: true
I20260810 01:57:32.327047 23271 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58158:
name: "default.Table_bb2d4e224bd14be09d60bce7977d4c19"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:32.324028 23273 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58158:
name: "default.Table_1bd23f5807da48238cbddc7597b8ab50"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:32.331552 23276 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58158:
table { table_name: "default.Table_22726fd1a11f409f9f5287ab16d8e551" } modify_external_catalogs: true
I20260810 01:57:32.326392 23274 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58158:
name: "default.Table_5a4f0e1bcc16485ab9a7148d13d21a68"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:32.331435 23280 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58158:
name: "default.Table_14c5bada100e48adb5f378e73ee7d00b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260810 01:57:32.333410 23584 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 10 received events: NotificationEvent(eventId=1, eventTime=1786327049, eventType=CREATE_TABLE, dbName=default, tableName=table_4fb63c687c11493ca7c0b76735f137e9, message={"server":"","servicePrincipal":"","db":"default","table":"table_4fb63c687c11493ca7c0b76735f137e9","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_4fb63c687c11493ca7c0b76735f137e9\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1786327049},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/warehouse/table_4fb63c687c11493ca7c0b76735f137e9\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_4fb63c687c11493ca7c0b76735f137e9\",\"kudu.table_id\":\"cb924727e6014a2a9dd271f962df3807\",\"transient_lastDdlTime\":\"1786327049\",\"comment\":\"\",\"kudu.master_addresses\":\"127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"dfa48c56b63b4d0cbf3d6124921e8a17\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1786327049,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1786327049, eventType=CREATE_TABLE, dbName=default, tableName=table_c8ad906be0f74210912a57adead21255, message={"server":"","servicePrincipal":"","db":"default","table":"table_c8ad906be0f74210912a57adead21255","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_c8ad906be0f74210912a57adead21255\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1786327049},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/warehouse/table_c8ad906be0f74210912a57adead21255\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_c8ad906be0f74210912a57adead21255\",\"kudu.table_id\":\"4da3c920ea6a48588c9f0c91da00fb69\",\"transient_lastDdlTime\":\"1786327049\",\"comment\":\"\",\"kudu.master_addresses\":\"127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"dfa48c56b63b4d0cbf3d6124921e8a17\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1786327049,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
@ 0x7fc26a01fd5d google::LogMessage::Fail() at ??:0
@ 0x7fc26a023b23 google::LogMessage::SendToLog() at ??:0
I20260810 01:57:32.338703 24631 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 3 FOLLOWER]: Refusing update from remote peer 919313c0dee540a690f46dbc8208d20b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 34. (index mismatch)
@ 0x7fc26a01f75c google::LogMessage::Flush() at ??:0
I20260810 01:57:32.340058 24749 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [LEADER]: Connected to new peer: Peer: permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 4, Time since last communication: 0.000s
@ 0x7fc26a01f9a1 google::LogMessage::~LogMessage() at ??:0
W20260810 01:57:32.347249 23260 consensus_peers.cc:597] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b -> Peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Couldn't send request to peer b1422349180c446db6c1768b3025253c. Status: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 01:57:32.348830 24564 consensus_queue.cc:1048] T 2230d919953e49208659450a024c6a62 P 498d352dbea9405199a121e84a4ab926 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
@ 0x7fc280ce8f64 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7fc280ce7811 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7fc280cef48c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260810 01:57:32.375994 24493 consensus_queue.cc:1048] T 2230d919953e49208659450a024c6a62 P 498d352dbea9405199a121e84a4ab926 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
@ 0x7fc280cef309 std::_Function_handler<>::_M_invoke() at ??:0
@ 0xaaf86d std::function<>::operator()() at ??:?
@ 0x7fc26cd47dd9 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fc2704176db start_thread at ??:0
@ 0x7fc268ece61f clone at ??:0
I20260810 01:57:32.404100 24764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "919313c0dee540a690f46dbc8208d20b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:32.404485 24764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:57:32.409319 24736 sys_catalog.cc:455] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 919313c0dee540a690f46dbc8208d20b. Latest consensus state: current_term: 3 leader_uuid: "919313c0dee540a690f46dbc8208d20b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:32.409574 24736 sys_catalog.cc:458] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:57:32.413463 24758 mvcc.cc:204] Tried to move back new op lower bound from 7316795601032024064 to 7316795595448557568. Current Snapshot: MvccSnapshot[applied={T|T < 7316795601032024064}]
I20260810 01:57:32.462859 24493 consensus_queue.cc:1048] T 89d9556cc526444299c786de606bb5b1 P 498d352dbea9405199a121e84a4ab926 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:57:32.468384 24438 consensus_queue.cc:1048] T 89d9556cc526444299c786de606bb5b1 P 498d352dbea9405199a121e84a4ab926 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:57:32.495978 24490 consensus_queue.cc:1048] T a48e9f780b4e49abafaadbd78ab8711a P 48e33bc1e8e94d4887bb64214d0b6c36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:57:32.502853 24490 consensus_queue.cc:1048] T a48e9f780b4e49abafaadbd78ab8711a P 48e33bc1e8e94d4887bb64214d0b6c36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 01:57:32.583472 24321 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.22.69.62:35311)
W20260810 01:57:32.583454 24572 connection.cc:573] server connection from 127.0.0.1:56266 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 01:57:32.583765 24321 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.22.69.62:35311)
W20260810 01:57:32.584452 24321 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.22.69.62:35311)
W20260810 01:57:32.584911 24321 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.22.69.62:35311)
I20260810 01:57:32.845109 24736 raft_consensus.cc:493] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 919313c0dee540a690f46dbc8208d20b)
I20260810 01:57:32.845319 24736 raft_consensus.cc:515] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:32.845964 24736 leader_election.cc:290] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), b1422349180c446db6c1768b3025253c (127.22.69.61:41945)
W20260810 01:57:32.848181 24574 leader_election.cc:336] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 01:57:32.848388 24577 leader_election.cc:336] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311): Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111)
I20260810 01:57:32.848548 24577 leader_election.cc:304] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 757bb3f336084510845ed58df264c721; no voters: 919313c0dee540a690f46dbc8208d20b, b1422349180c446db6c1768b3025253c
I20260810 01:57:32.848780 24736 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260810 01:57:33.458097 24823 raft_consensus.cc:493] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 919313c0dee540a690f46dbc8208d20b)
I20260810 01:57:33.458263 24823 raft_consensus.cc:515] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:33.458822 24823 leader_election.cc:290] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), b1422349180c446db6c1768b3025253c (127.22.69.61:41945)
W20260810 01:57:33.460748 24577 leader_election.cc:336] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311): Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111)
W20260810 01:57:33.460925 24574 leader_election.cc:336] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
I20260810 01:57:33.461028 24574 leader_election.cc:304] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 757bb3f336084510845ed58df264c721; no voters: 919313c0dee540a690f46dbc8208d20b, b1422349180c446db6c1768b3025253c
I20260810 01:57:33.461300 24823 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260810 01:57:34.293630 22804 external_mini_cluster.cc:1372] Running /tmp/dist-test-task_qhVVJ/build/asan/bin/kudu
/tmp/dist-test-task_qhVVJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.69.61:41945
--webserver_interface=127.22.69.61
--webserver_port=33591
--builtin_ntp_servers=127.22.69.20:43209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917
--hive_metastore_uris=thrift://127.0.0.1:37513
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260810 01:57:34.295401 24839 raft_consensus.cc:493] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 919313c0dee540a690f46dbc8208d20b)
I20260810 01:57:34.295578 24839 raft_consensus.cc:515] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:34.296280 24839 leader_election.cc:290] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), b1422349180c446db6c1768b3025253c (127.22.69.61:41945)
W20260810 01:57:34.298461 24577 leader_election.cc:336] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311): Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111)
W20260810 01:57:34.298668 24574 leader_election.cc:336] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
I20260810 01:57:34.298799 24574 leader_election.cc:304] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 757bb3f336084510845ed58df264c721; no voters: 919313c0dee540a690f46dbc8208d20b, b1422349180c446db6c1768b3025253c
I20260810 01:57:34.299105 24839 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260810 01:57:34.641688 24845 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:57:34.642022 24845 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:57:34.642165 24845 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260810 01:57:34.650910 24845 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:57:34.650992 24845 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:57:34.651124 24845 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:57:34.651207 24845 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:57:34.661300 24845 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.69.20:43209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:37513
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.69.61:41945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.69.61
--webserver_port=33591
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 10 Aug 2026 01:44:04 UTC on e7f111948823
build id 13266
ASAN enabled
I20260810 01:57:34.662887 24845 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:57:34.664718 24845 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260810 01:57:34.673270 24857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:57:34.673455 24856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:57:34.674549 24860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:57:34.674927 24845 server_base.cc:1061] running on GCE node
I20260810 01:57:34.675715 24845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:57:34.677074 24845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:57:34.678282 24845 hybrid_clock.cc:648] HybridClock initialized: now 1786327054678239 us; error 38 us; skew 500 ppm
I20260810 01:57:34.678663 24845 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 01:57:34.680877 24845 webserver.cc:533] Webserver started at http://127.22.69.61:33591/ using document root <none> and password file <none>
I20260810 01:57:34.681587 24845 fs_manager.cc:362] Metadata directory not provided
I20260810 01:57:34.681696 24845 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:57:34.687143 24845 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.001s sys 0.002s
I20260810 01:57:34.689989 24868 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:57:34.691416 24845 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 01:57:34.691563 24845 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/data,/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/wal
uuid: "b1422349180c446db6c1768b3025253c"
format_stamp: "Formatted at 2026-08-10 01:57:26 on dist-test-slave-n326"
I20260810 01:57:34.692239 24845 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:57:34.721481 24845 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260810 01:57:34.722383 24845 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:57:34.722692 24845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:57:34.744186 24845 rpc_server.cc:307] RPC server started. Bound to: 127.22.69.61:41945
I20260810 01:57:34.744205 24957 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.22.69.61:41945 every 8 connection(s)
I20260810 01:57:34.745916 24845 server_base.cc:1193] Dumped server information to /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/data/info.pb
I20260810 01:57:34.753232 24958 sys_catalog.cc:263] Verifying existing consensus state
I20260810 01:57:34.755435 22804 external_mini_cluster.cc:1434] Started /tmp/dist-test-task_qhVVJ/build/asan/bin/kudu as pid 24845
I20260810 01:57:34.757117 24958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c: Bootstrap starting.
I20260810 01:57:34.769164 24958 log.cc:826] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c: Log is configured to *not* fsync() on all Append() calls
I20260810 01:57:34.797118 24958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c: Bootstrap replayed 1/1 log segments. Stats: ops{read=32 overwritten=0 applied=32 ignored=0} inserts{seen=24 ignored=0} mutations{seen=19 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 01:57:34.797762 24958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c: Bootstrap complete.
I20260810 01:57:34.805626 24958 raft_consensus.cc:359] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:34.806624 24958 raft_consensus.cc:740] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1422349180c446db6c1768b3025253c, State: Initialized, Role: FOLLOWER
I20260810 01:57:34.807315 24958 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 32, Last appended: 2.32, Last appended by leader: 32, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:34.808239 24965 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:34.808749 24965 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:57:34.809629 24958 sys_catalog.cc:565] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:57:34.821134 24986 catalog_manager.cc:1295] Loaded cluster ID: dfa48c56b63b4d0cbf3d6124921e8a17
I20260810 01:57:34.821300 24986 catalog_manager.cc:1588] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c: loading cluster ID for follower catalog manager: success
I20260810 01:57:34.825639 24986 catalog_manager.cc:1610] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c: acquiring CA information for follower catalog manager: success
I20260810 01:57:34.828531 24986 catalog_manager.cc:1638] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 01:57:34.828394 24845 master_runner.cc:187] Error getting master registration for 127.22.69.62:35311: Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111),
I20260810 01:57:34.834234 24845 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 01:57:34.956298 24069 heartbeater.cc:344] Connected to a master server at 127.22.69.61:41945
I20260810 01:57:34.960428 24900 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" instance_seqno: 1786327048775850) as {username='slave'} at 127.22.69.2:36305; Asking this server to re-register.
I20260810 01:57:34.961257 24069 heartbeater.cc:461] Registering TS with master...
I20260810 01:57:34.962879 24900 ts_manager.cc:194] Registered new tserver with Master: 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
I20260810 01:57:35.023118 23846 heartbeater.cc:344] Connected to a master server at 127.22.69.61:41945
I20260810 01:57:35.025138 24900 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" instance_seqno: 1786327048176732) as {username='slave'} at 127.22.69.1:51749; Asking this server to re-register.
I20260810 01:57:35.025781 23846 heartbeater.cc:461] Registering TS with master...
I20260810 01:57:35.027048 24900 ts_manager.cc:194] Registered new tserver with Master: 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491)
I20260810 01:57:35.183151 24965 raft_consensus.cc:493] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:57:35.183485 24965 raft_consensus.cc:515] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:35.185485 24965 leader_election.cc:290] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:57:35.186205 24872 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111)
W20260810 01:57:35.189642 24872 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311): Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111)
I20260810 01:57:35.194200 24631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b1422349180c446db6c1768b3025253c" candidate_term: 3 candidate_status { last_received { term: 2 index: 32 } } ignore_live_leader: false dest_uuid: "757bb3f336084510845ed58df264c721" is_pre_election: true
I20260810 01:57:35.194496 24631 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1422349180c446db6c1768b3025253c in current term 3: Already voted for candidate 919313c0dee540a690f46dbc8208d20b in this term.
I20260810 01:57:35.195391 24870 leader_election.cc:304] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1422349180c446db6c1768b3025253c; no voters: 757bb3f336084510845ed58df264c721, 919313c0dee540a690f46dbc8208d20b
I20260810 01:57:35.195839 24965 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 2 FOLLOWER]: Advancing to term 3
I20260810 01:57:35.200970 24965 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260810 01:57:35.218454 24301 heartbeater.cc:344] Connected to a master server at 127.22.69.61:41945
I20260810 01:57:35.220436 24900 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "498d352dbea9405199a121e84a4ab926" instance_seqno: 1786327049319804) as {username='slave'} at 127.22.69.3:39281; Asking this server to re-register.
I20260810 01:57:35.221046 24301 heartbeater.cc:461] Registering TS with master...
I20260810 01:57:35.222270 24900 ts_manager.cc:194] Registered new tserver with Master: 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
I20260810 01:57:35.734861 25021 raft_consensus.cc:493] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 919313c0dee540a690f46dbc8208d20b)
I20260810 01:57:35.735111 25021 raft_consensus.cc:515] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:35.735778 25021 leader_election.cc:290] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), b1422349180c446db6c1768b3025253c (127.22.69.61:41945)
W20260810 01:57:35.738034 24577 leader_election.cc:336] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311): Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111)
I20260810 01:57:35.742332 24920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "757bb3f336084510845ed58df264c721" candidate_term: 4 candidate_status { last_received { term: 3 index: 34 } } ignore_live_leader: false dest_uuid: "b1422349180c446db6c1768b3025253c" is_pre_election: true
I20260810 01:57:35.742718 24920 raft_consensus.cc:2468] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 757bb3f336084510845ed58df264c721 in term 3.
I20260810 01:57:35.743212 24574 leader_election.cc:304] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c; no voters: 919313c0dee540a690f46dbc8208d20b
I20260810 01:57:35.743518 25021 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260810 01:57:35.743650 25021 raft_consensus.cc:493] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 919313c0dee540a690f46dbc8208d20b)
I20260810 01:57:35.743762 25021 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 3 FOLLOWER]: Advancing to term 4
I20260810 01:57:35.748217 25021 raft_consensus.cc:515] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:35.748876 25021 leader_election.cc:290] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [CANDIDATE]: Term 4 election: Requested vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), b1422349180c446db6c1768b3025253c (127.22.69.61:41945)
I20260810 01:57:35.749382 24920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "757bb3f336084510845ed58df264c721" candidate_term: 4 candidate_status { last_received { term: 3 index: 34 } } ignore_live_leader: false dest_uuid: "b1422349180c446db6c1768b3025253c"
I20260810 01:57:35.749571 24920 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 3 FOLLOWER]: Advancing to term 4
W20260810 01:57:35.750675 24577 leader_election.cc:336] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311): Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111)
I20260810 01:57:35.752204 24920 raft_consensus.cc:2468] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 757bb3f336084510845ed58df264c721 in term 4.
I20260810 01:57:35.752602 24574 leader_election.cc:304] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c; no voters: 919313c0dee540a690f46dbc8208d20b
I20260810 01:57:35.752884 25021 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 4 FOLLOWER]: Leader election won for term 4
I20260810 01:57:35.753268 25021 raft_consensus.cc:697] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 4 LEADER]: Becoming Leader. State: Replica: 757bb3f336084510845ed58df264c721, State: Running, Role: LEADER
I20260810 01:57:35.753739 25021 consensus_queue.cc:237] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 3.34, Last appended by leader: 34, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:35.755496 25027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 757bb3f336084510845ed58df264c721. Latest consensus state: current_term: 4 leader_uuid: "757bb3f336084510845ed58df264c721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:35.755790 25027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [sys.catalog]: This master's current role is: LEADER
I20260810 01:57:36.200331 24920 raft_consensus.cc:1275] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 4 FOLLOWER]: Refusing update from remote peer 757bb3f336084510845ed58df264c721: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 4 index: 35. (index mismatch)
I20260810 01:57:36.206532 25027 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 32, Time since last communication: 0.000s
I20260810 01:57:36.213646 25042 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 757bb3f336084510845ed58df264c721. Latest consensus state: current_term: 4 leader_uuid: "757bb3f336084510845ed58df264c721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:36.213903 25042 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: This master's current role is: FOLLOWER
W20260810 01:57:36.221865 24577 consensus_peers.cc:597] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 -> Peer 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311): Couldn't send request to peer 919313c0dee540a690f46dbc8208d20b. Status: Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 01:57:36.223162 25027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "757bb3f336084510845ed58df264c721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:36.223405 25027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [sys.catalog]: This master's current role is: LEADER
I20260810 01:57:36.223646 25027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "757bb3f336084510845ed58df264c721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:36.223879 25027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [sys.catalog]: This master's current role is: LEADER
I20260810 01:57:36.225374 25061 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "757bb3f336084510845ed58df264c721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:36.225595 25042 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "757bb3f336084510845ed58df264c721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:36.225836 25061 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:57:36.225872 25042 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:57:36.272379 25030 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260810 01:57:36.276101 25030 catalog_manager.cc:679] Loaded metadata for table default.table_72b6f93a124042e4a090dc194134c75a [id=017756566b72415aa33842f8df2b3e91]
I20260810 01:57:36.276826 25030 catalog_manager.cc:679] Loaded metadata for table default.table_73d521a2c5504815ad6d7cc3dd0bf818 [id=2f1832807601444a88d61b8396a67db9]
I20260810 01:57:36.277411 25030 catalog_manager.cc:679] Loaded metadata for table default.table_c8ad906be0f74210912a57adead21255 [id=4da3c920ea6a48588c9f0c91da00fb69]
I20260810 01:57:36.277992 25030 catalog_manager.cc:679] Loaded metadata for table default.table_983bf2e604df4fdbac75c5aca292f736 [id=552771dde220474a9be9abeefd3a0472]
I20260810 01:57:36.278509 25030 catalog_manager.cc:679] Loaded metadata for table default.table_22726fd1a11f409f9f5287ab16d8e551 [id=5edf9cb336414ae3bd41e9f5ca41fcca]
I20260810 01:57:36.278898 25030 catalog_manager.cc:679] Loaded metadata for table default.table_0ba2046f969346f68f1bde7cf6c9f273 [id=630b03060b7646118927a18573c0920c]
I20260810 01:57:36.279320 25030 catalog_manager.cc:679] Loaded metadata for table default.table_510ae1b5fcc74db788ffb64925cad250 [id=7d240d10089c47eaa014d59c4ac7f0b4]
I20260810 01:57:36.279627 25030 catalog_manager.cc:679] Loaded metadata for table default.table_379ea52611ef49d591a022addb7afc0c [id=87ba140e9080402aa494198d6f212f05]
I20260810 01:57:36.279893 25030 catalog_manager.cc:679] Loaded metadata for table default.table_57697192c9bf44969c2b200e94865a3e [id=ef2e836fffc64aa6a8ddc0425f39e317]
I20260810 01:57:36.281682 25030 tablet_loader.cc:96] loaded metadata for tablet 09ed68395e2d44a5b51a22abe2203406 (table default.table_73d521a2c5504815ad6d7cc3dd0bf818 [id=2f1832807601444a88d61b8396a67db9])
I20260810 01:57:36.282418 25030 tablet_loader.cc:96] loaded metadata for tablet 16db112abfb34d4a9f2ecbee98c39d8b (table default.table_72b6f93a124042e4a090dc194134c75a [id=017756566b72415aa33842f8df2b3e91])
I20260810 01:57:36.282902 25030 tablet_loader.cc:96] loaded metadata for tablet 1c3db8429f9f4cf5b29d3d1a6cc0651c (table default.table_379ea52611ef49d591a022addb7afc0c [id=87ba140e9080402aa494198d6f212f05])
I20260810 01:57:36.283236 25030 tablet_loader.cc:96] loaded metadata for tablet 2230d919953e49208659450a024c6a62 (table default.table_0ba2046f969346f68f1bde7cf6c9f273 [id=630b03060b7646118927a18573c0920c])
I20260810 01:57:36.283627 25030 tablet_loader.cc:96] loaded metadata for tablet 4579a90be0b6409781fd9e470324192c (table default.table_22726fd1a11f409f9f5287ab16d8e551 [id=5edf9cb336414ae3bd41e9f5ca41fcca])
I20260810 01:57:36.283931 25030 tablet_loader.cc:96] loaded metadata for tablet 5c49a207b37e40579d34002c3fb57afb (table default.table_c8ad906be0f74210912a57adead21255 [id=4da3c920ea6a48588c9f0c91da00fb69])
I20260810 01:57:36.284255 25030 tablet_loader.cc:96] loaded metadata for tablet 89d9556cc526444299c786de606bb5b1 (table default.table_510ae1b5fcc74db788ffb64925cad250 [id=7d240d10089c47eaa014d59c4ac7f0b4])
I20260810 01:57:36.284587 25030 tablet_loader.cc:96] loaded metadata for tablet a48e9f780b4e49abafaadbd78ab8711a (table default.table_57697192c9bf44969c2b200e94865a3e [id=ef2e836fffc64aa6a8ddc0425f39e317])
I20260810 01:57:36.285205 25030 tablet_loader.cc:96] loaded metadata for tablet e6ef3afe40a847dbb08792578d52401d (table default.table_983bf2e604df4fdbac75c5aca292f736 [id=552771dde220474a9be9abeefd3a0472])
I20260810 01:57:36.285456 25030 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260810 01:57:36.286405 25030 catalog_manager.cc:1295] Loaded cluster ID: dfa48c56b63b4d0cbf3d6124921e8a17
I20260810 01:57:36.286501 25030 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260810 01:57:36.288036 25030 catalog_manager.cc:1540] Loading token signing keys...
I20260810 01:57:36.288888 25030 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721: Loaded TSK: 0
I20260810 01:57:36.289731 25030 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260810 01:57:36.290560 25030 catalog_manager.cc:1563] Loading latest processed Hive Metastore notification log event ID...
I20260810 01:57:36.291445 25030 catalog_manager.cc:5941] Last processed Hive Metastore notification event ID: -1
I20260810 01:57:36.353852 24614 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56258:
name: "default.Table_73822ba0bac443d19cc7c66f6f547bda"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:36.354079 24616 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:56258:
table { table_name: "default.Table_72b6f93a124042e4a090dc194134c75a" } new_table_name: "default.Table_b8900cd2c2574605ade34fc7ccc46f94" modify_external_catalogs: true
I20260810 01:57:36.354135 24610 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:56258:
table { table_name: "default.Table_379ea52611ef49d591a022addb7afc0c" } new_table_name: "default.Table_09a7f5150ded428991a5d464657e85ae" modify_external_catalogs: true
I20260810 01:57:36.355007 24609 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:56258:
table { table_name: "default.Table_c8ad906be0f74210912a57adead21255" } new_table_name: "default.Table_e2cb378ea960473c833b1d18fbe32f91" modify_external_catalogs: true
I20260810 01:57:36.355280 24606 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:56258:
table { table_name: "default.Table_73d521a2c5504815ad6d7cc3dd0bf818" } new_table_name: "default.Table_f5851b3fa264482388e4e50f34d4e37c" modify_external_catalogs: true
I20260810 01:57:36.355721 24595 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:56258:
table { table_name: "default.Table_983bf2e604df4fdbac75c5aca292f736" } new_table_name: "default.Table_df285e759e764cdeb38de2aad0d1873d" modify_external_catalogs: true
I20260810 01:57:36.355895 24596 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56258:
name: "default.Table_9175c2bd9b0542f2a6b95eae8a882b7a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:36.356416 24588 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56258:
table { table_name: "default.Table_0ba2046f969346f68f1bde7cf6c9f273" } modify_external_catalogs: true
I20260810 01:57:36.356492 24600 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56258:
name: "default.Table_70288f170151490fb6ba63a3bf29f01e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:36.356680 24588 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56258:
table { table_name: "default.Table_0ba2046f969346f68f1bde7cf6c9f273" } modify_external_catalogs: true
I20260810 01:57:36.357299 24592 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56258:
table { table_name: "default.Table_57697192c9bf44969c2b200e94865a3e" } modify_external_catalogs: true
I20260810 01:57:36.357546 24592 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56258:
table { table_name: "default.Table_57697192c9bf44969c2b200e94865a3e" } modify_external_catalogs: true
F20260810 01:57:36.361510 24716 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 10 received events: NotificationEvent(eventId=1, eventTime=1786327049, eventType=CREATE_TABLE, dbName=default, tableName=table_4fb63c687c11493ca7c0b76735f137e9, message={"server":"","servicePrincipal":"","db":"default","table":"table_4fb63c687c11493ca7c0b76735f137e9","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_4fb63c687c11493ca7c0b76735f137e9\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1786327049},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/warehouse/table_4fb63c687c11493ca7c0b76735f137e9\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_4fb63c687c11493ca7c0b76735f137e9\",\"kudu.table_id\":\"cb924727e6014a2a9dd271f962df3807\",\"transient_lastDdlTime\":\"1786327049\",\"comment\":\"\",\"kudu.master_addresses\":\"127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"dfa48c56b63b4d0cbf3d6124921e8a17\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1786327049,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1786327049, eventType=CREATE_TABLE, dbName=default, tableName=table_c8ad906be0f74210912a57adead21255, message={"server":"","servicePrincipal":"","db":"default","table":"table_c8ad906be0f74210912a57adead21255","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_c8ad906be0f74210912a57adead21255\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1786327049},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/warehouse/table_c8ad906be0f74210912a57adead21255\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_c8ad906be0f74210912a57adead21255\",\"kudu.table_id\":\"4da3c920ea6a48588c9f0c91da00fb69\",\"transient_lastDdlTime\":\"1786327049\",\"comment\":\"\",\"kudu.master_addresses\":\"127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"dfa48c56b63b4d0cbf3d6124921e8a17\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1786327049,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
@ 0x7f70a1c74d5d google::LogMessage::Fail() at ??:0
@ 0x7f70a1c78b23 google::LogMessage::SendToLog() at ??:0
@ 0x7f70a1c7475c google::LogMessage::Flush() at ??:0
@ 0x7f70a1c749a1 google::LogMessage::~LogMessage() at ??:0
@ 0x7f70b893df64 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7f70b893c811 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f70b894448c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7f70b8944309 std::_Function_handler<>::_M_invoke() at ??:0
@ 0xaaf86d std::function<>::operator()() at ??:?
@ 0x7f70a499cdd9 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f70a806c6db start_thread at ??:0
@ 0x7f70a0b2361f clone at ??:0
W20260810 01:57:36.563884 23681 connection.cc:573] client connection to 127.22.69.60:36917 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 01:57:36.565182 23890 connection.cc:573] client connection to 127.22.69.60:36917 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 01:57:36.565546 24317 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.22.69.60:36917)
W20260810 01:57:36.565861 24317 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.22.69.60:36917)
W20260810 01:57:36.566146 24317 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.22.69.60:36917)
W20260810 01:57:36.566886 24317 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.22.69.60:36917)
I20260810 01:57:36.716003 25042 raft_consensus.cc:493] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 757bb3f336084510845ed58df264c721)
I20260810 01:57:36.716169 25042 raft_consensus.cc:515] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:36.716876 25042 leader_election.cc:290] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:57:36.719494 24870 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
W20260810 01:57:36.719828 24872 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311): Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111)
I20260810 01:57:36.720017 24872 leader_election.cc:304] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1422349180c446db6c1768b3025253c; no voters: 757bb3f336084510845ed58df264c721, 919313c0dee540a690f46dbc8208d20b
I20260810 01:57:36.720352 25042 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260810 01:57:37.081235 22804 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-task_qhVVJ/build/asan/bin/kudu pid 24845
I20260810 01:57:37.093317 24953 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 01:57:37.212322 22804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task_qhVVJ/build/asan/bin/kudu with pid 24845
W20260810 01:57:37.244364 24111 connection.cc:573] client connection to 127.22.69.61:41945 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 01:57:37.973196 22804 external_mini_cluster.cc:1372] Running /tmp/dist-test-task_qhVVJ/build/asan/bin/kudu
/tmp/dist-test-task_qhVVJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.69.61:41945
--webserver_interface=127.22.69.61
--webserver_port=33591
--builtin_ntp_servers=127.22.69.20:43209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917
--hive_metastore_uris=thrift://127.0.0.1:37513
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260810 01:57:38.320221 25117 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:57:38.320535 25117 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:57:38.320636 25117 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260810 01:57:38.328804 25117 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:57:38.328881 25117 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:57:38.328955 25117 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:57:38.329025 25117 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:57:38.343623 25117 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.69.20:43209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:37513
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.69.61:41945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.69.61
--webserver_port=33591
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 10 Aug 2026 01:44:04 UTC on e7f111948823
build id 13266
ASAN enabled
I20260810 01:57:38.345472 25117 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:57:38.347638 25117 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260810 01:57:38.356777 25127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:57:38.356618 25130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:57:38.357081 25132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:57:38.357687 25117 server_base.cc:1061] running on GCE node
I20260810 01:57:38.358355 25117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:57:38.359678 25117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:57:38.360940 25117 hybrid_clock.cc:648] HybridClock initialized: now 1786327058360881 us; error 40 us; skew 500 ppm
I20260810 01:57:38.361384 25117 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 01:57:38.364354 25117 webserver.cc:533] Webserver started at http://127.22.69.61:33591/ using document root <none> and password file <none>
I20260810 01:57:38.365038 25117 fs_manager.cc:362] Metadata directory not provided
I20260810 01:57:38.365131 25117 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:57:38.370450 25117 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 01:57:38.373205 25139 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:57:38.374446 25117 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 01:57:38.374627 25117 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/data,/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/wal
uuid: "b1422349180c446db6c1768b3025253c"
format_stamp: "Formatted at 2026-08-10 01:57:26 on dist-test-slave-n326"
I20260810 01:57:38.375357 25117 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:57:38.395426 25117 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260810 01:57:38.396219 25117 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:57:38.396533 25117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:57:38.417148 25117 rpc_server.cc:307] RPC server started. Bound to: 127.22.69.61:41945
I20260810 01:57:38.417167 25225 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.22.69.61:41945 every 8 connection(s)
I20260810 01:57:38.418812 25117 server_base.cc:1193] Dumped server information to /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/data/info.pb
I20260810 01:57:38.424144 22804 external_mini_cluster.cc:1434] Started /tmp/dist-test-task_qhVVJ/build/asan/bin/kudu as pid 25117
I20260810 01:57:38.427738 25226 sys_catalog.cc:263] Verifying existing consensus state
I20260810 01:57:38.436344 25226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c: Bootstrap starting.
I20260810 01:57:38.449767 25226 log.cc:826] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c: Log is configured to *not* fsync() on all Append() calls
I20260810 01:57:38.478739 25226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c: Bootstrap replayed 1/1 log segments. Stats: ops{read=35 overwritten=0 applied=35 ignored=0} inserts{seen=25 ignored=0} mutations{seen=20 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 01:57:38.479467 25226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c: Bootstrap complete.
I20260810 01:57:38.481683 24069 heartbeater.cc:344] Connected to a master server at 127.22.69.61:41945
I20260810 01:57:38.485328 25226 raft_consensus.cc:359] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 4 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:38.486009 25226 raft_consensus.cc:740] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1422349180c446db6c1768b3025253c, State: Initialized, Role: FOLLOWER
I20260810 01:57:38.486598 25226 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 35, Last appended: 4.35, Last appended by leader: 35, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:38.487422 25232 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 4 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:38.487689 25232 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:57:38.488370 25226 sys_catalog.cc:565] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:57:38.502569 25257 catalog_manager.cc:1295] Loaded cluster ID: dfa48c56b63b4d0cbf3d6124921e8a17
I20260810 01:57:38.502735 25257 catalog_manager.cc:1588] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c: loading cluster ID for follower catalog manager: success
I20260810 01:57:38.498312 25117 master_runner.cc:187] Error getting master registration for 127.22.69.62:35311: Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111),
I20260810 01:57:38.506944 25117 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 01:57:38.507388 25257 catalog_manager.cc:1610] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c: acquiring CA information for follower catalog manager: success
I20260810 01:57:38.510043 25257 catalog_manager.cc:1638] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 01:57:38.545607 23846 heartbeater.cc:344] Connected to a master server at 127.22.69.61:41945
I20260810 01:57:38.547685 25170 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" instance_seqno: 1786327048176732) as {username='slave'} at 127.22.69.1:42803; Asking this server to re-register.
I20260810 01:57:38.548322 23846 heartbeater.cc:461] Registering TS with master...
I20260810 01:57:38.549997 25170 ts_manager.cc:194] Registered new tserver with Master: 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491)
I20260810 01:57:38.753276 24301 heartbeater.cc:344] Connected to a master server at 127.22.69.61:41945
I20260810 01:57:38.754689 25170 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "498d352dbea9405199a121e84a4ab926" instance_seqno: 1786327049319804) as {username='slave'} at 127.22.69.3:40577; Asking this server to re-register.
I20260810 01:57:38.755362 24301 heartbeater.cc:461] Registering TS with master...
I20260810 01:57:38.756914 25170 ts_manager.cc:194] Registered new tserver with Master: 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
I20260810 01:57:38.862551 25232 raft_consensus.cc:493] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:57:38.862829 25232 raft_consensus.cc:515] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:38.864955 25232 leader_election.cc:290] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:57:38.865700 25145 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111)
W20260810 01:57:38.867642 25142 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
W20260810 01:57:38.867905 25145 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311): Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111)
I20260810 01:57:38.868057 25145 leader_election.cc:304] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1422349180c446db6c1768b3025253c; no voters: 757bb3f336084510845ed58df264c721, 919313c0dee540a690f46dbc8208d20b
I20260810 01:57:38.868400 25232 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260810 01:57:38.985333 25170 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" instance_seqno: 1786327048775850) as {username='slave'} at 127.22.69.2:43247; Asking this server to re-register.
I20260810 01:57:38.985926 24069 heartbeater.cc:461] Registering TS with master...
I20260810 01:57:38.987221 25170 ts_manager.cc:194] Registered new tserver with Master: 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
I20260810 01:57:39.419597 25269 raft_consensus.cc:493] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:57:39.419808 25269 raft_consensus.cc:515] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:39.420589 25269 leader_election.cc:290] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:57:39.429070 25145 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311): Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111)
W20260810 01:57:39.429248 25142 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:57:39.429404 25142 leader_election.cc:304] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1422349180c446db6c1768b3025253c; no voters: 757bb3f336084510845ed58df264c721, 919313c0dee540a690f46dbc8208d20b
I20260810 01:57:39.429701 25269 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260810 01:57:40.041481 25278 raft_consensus.cc:493] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:57:40.041673 25278 raft_consensus.cc:515] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:40.042250 25278 leader_election.cc:290] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:57:40.044494 25145 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311): Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111)
W20260810 01:57:40.045039 25142 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:57:40.045192 25142 leader_election.cc:304] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1422349180c446db6c1768b3025253c; no voters: 757bb3f336084510845ed58df264c721, 919313c0dee540a690f46dbc8208d20b
I20260810 01:57:40.045430 25278 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260810 01:57:41.404619 25298 raft_consensus.cc:493] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:57:41.404820 25298 raft_consensus.cc:515] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:41.405462 25298 leader_election.cc:290] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:57:41.407593 25145 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311): Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111)
W20260810 01:57:41.407748 25142 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:57:41.407902 25142 leader_election.cc:304] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1422349180c446db6c1768b3025253c; no voters: 757bb3f336084510845ed58df264c721, 919313c0dee540a690f46dbc8208d20b
I20260810 01:57:41.408169 25298 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260810 01:57:41.588888 22804 external_mini_cluster.cc:1372] Running /tmp/dist-test-task_qhVVJ/build/asan/bin/kudu
/tmp/dist-test-task_qhVVJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.69.60:36917
--webserver_interface=127.22.69.60
--webserver_port=44467
--builtin_ntp_servers=127.22.69.20:43209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917
--hive_metastore_uris=thrift://127.0.0.1:37513
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260810 01:57:41.937431 25302 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:57:41.937737 25302 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:57:41.937822 25302 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260810 01:57:41.946835 25302 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:57:41.946919 25302 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:57:41.946973 25302 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:57:41.947021 25302 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:57:41.956770 25302 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.69.20:43209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:37513
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.69.60:36917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.69.60
--webserver_port=44467
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 10 Aug 2026 01:44:04 UTC on e7f111948823
build id 13266
ASAN enabled
I20260810 01:57:41.958292 25302 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:57:41.960104 25302 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260810 01:57:41.965845 25323 raft_consensus.cc:493] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:57:41.966049 25323 raft_consensus.cc:515] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:41.967404 25323 leader_election.cc:290] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:57:41.967594 25320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:57:41.971094 25322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:57:41.971760 25142 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
W20260810 01:57:41.972548 25145 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311): Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111)
I20260810 01:57:41.972690 25145 leader_election.cc:304] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1422349180c446db6c1768b3025253c; no voters: 757bb3f336084510845ed58df264c721, 919313c0dee540a690f46dbc8208d20b
I20260810 01:57:41.973026 25323 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260810 01:57:41.973284 25327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:57:41.973562 25302 server_base.cc:1061] running on GCE node
I20260810 01:57:41.974287 25302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:57:41.975713 25302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:57:41.977003 25302 hybrid_clock.cc:648] HybridClock initialized: now 1786327061976946 us; error 44 us; skew 500 ppm
I20260810 01:57:41.977418 25302 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 01:57:41.979635 25302 webserver.cc:533] Webserver started at http://127.22.69.60:44467/ using document root <none> and password file <none>
I20260810 01:57:41.980417 25302 fs_manager.cc:362] Metadata directory not provided
I20260810 01:57:41.980530 25302 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:57:41.985780 25302 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 01:57:41.988596 25342 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:57:41.989931 25302 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 01:57:41.990103 25302 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/data,/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/wal
uuid: "757bb3f336084510845ed58df264c721"
format_stamp: "Formatted at 2026-08-10 01:57:26 on dist-test-slave-n326"
I20260810 01:57:41.990777 25302 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:57:42.002323 25302 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260810 01:57:42.003136 25302 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:57:42.003399 25302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:57:42.023602 25302 rpc_server.cc:307] RPC server started. Bound to: 127.22.69.60:36917
I20260810 01:57:42.023608 25413 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.22.69.60:36917 every 8 connection(s)
I20260810 01:57:42.025281 25302 server_base.cc:1193] Dumped server information to /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/data/info.pb
I20260810 01:57:42.028498 22804 external_mini_cluster.cc:1434] Started /tmp/dist-test-task_qhVVJ/build/asan/bin/kudu as pid 25302
I20260810 01:57:42.032972 25414 sys_catalog.cc:263] Verifying existing consensus state
I20260810 01:57:42.038834 25414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721: Bootstrap starting.
I20260810 01:57:42.050155 25414 log.cc:826] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721: Log is configured to *not* fsync() on all Append() calls
I20260810 01:57:42.075624 25414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721: Bootstrap replayed 1/1 log segments. Stats: ops{read=35 overwritten=0 applied=35 ignored=0} inserts{seen=25 ignored=0} mutations{seen=20 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 01:57:42.076375 25414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721: Bootstrap complete.
I20260810 01:57:42.082450 23841 heartbeater.cc:344] Connected to a master server at 127.22.69.60:36917
I20260810 01:57:42.084215 24063 heartbeater.cc:344] Connected to a master server at 127.22.69.60:36917
I20260810 01:57:42.085256 25414 raft_consensus.cc:359] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 4 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:42.086069 25414 raft_consensus.cc:740] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: 757bb3f336084510845ed58df264c721, State: Initialized, Role: FOLLOWER
I20260810 01:57:42.086594 25414 consensus_queue.cc:260] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 35, Last appended: 4.35, Last appended by leader: 35, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:42.087371 25422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 4 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:42.087639 25422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:57:42.088356 25414 sys_catalog.cc:565] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:57:42.096632 25302 master_runner.cc:187] Error getting master registration for 127.22.69.62:35311: Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111),
I20260810 01:57:42.102330 25446 catalog_manager.cc:1295] Loaded cluster ID: dfa48c56b63b4d0cbf3d6124921e8a17
I20260810 01:57:42.102509 25446 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721: loading cluster ID for follower catalog manager: success
I20260810 01:57:42.102643 25302 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 01:57:42.106096 25446 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721: acquiring CA information for follower catalog manager: success
I20260810 01:57:42.108675 25446 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 01:57:42.249894 24298 heartbeater.cc:344] Connected to a master server at 127.22.69.60:36917
I20260810 01:57:42.252620 25360 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "498d352dbea9405199a121e84a4ab926" instance_seqno: 1786327049319804) as {username='slave'} at 127.22.69.3:46591; Asking this server to re-register.
I20260810 01:57:42.253365 24298 heartbeater.cc:461] Registering TS with master...
I20260810 01:57:42.255582 25360 ts_manager.cc:194] Registered new tserver with Master: 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
I20260810 01:57:42.586349 25360 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" instance_seqno: 1786327048176732) as {username='slave'} at 127.22.69.1:43411; Asking this server to re-register.
I20260810 01:57:42.586962 23841 heartbeater.cc:461] Registering TS with master...
I20260810 01:57:42.587003 25361 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" instance_seqno: 1786327048775850) as {username='slave'} at 127.22.69.2:43401; Asking this server to re-register.
I20260810 01:57:42.587692 24063 heartbeater.cc:461] Registering TS with master...
I20260810 01:57:42.588344 25361 ts_manager.cc:194] Registered new tserver with Master: 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491)
I20260810 01:57:42.589160 25360 ts_manager.cc:194] Registered new tserver with Master: 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
I20260810 01:57:42.682332 25464 raft_consensus.cc:493] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:57:42.682595 25464 raft_consensus.cc:515] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:42.684587 25464 leader_election.cc:290] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), b1422349180c446db6c1768b3025253c (127.22.69.61:41945)
W20260810 01:57:42.685972 25346 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111)
W20260810 01:57:42.687593 25346 leader_election.cc:336] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311): Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111)
I20260810 01:57:42.691737 25189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "757bb3f336084510845ed58df264c721" candidate_term: 5 candidate_status { last_received { term: 4 index: 35 } } ignore_live_leader: false dest_uuid: "b1422349180c446db6c1768b3025253c" is_pre_election: true
I20260810 01:57:42.692318 25189 raft_consensus.cc:2468] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 757bb3f336084510845ed58df264c721 in term 4.
I20260810 01:57:42.693145 25344 leader_election.cc:304] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c; no voters: 919313c0dee540a690f46dbc8208d20b
I20260810 01:57:42.693526 25464 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260810 01:57:42.693686 25464 raft_consensus.cc:493] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 01:57:42.693804 25464 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 4 FOLLOWER]: Advancing to term 5
I20260810 01:57:42.697750 25464 raft_consensus.cc:515] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:42.698503 25464 leader_election.cc:290] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [CANDIDATE]: Term 5 election: Requested vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), b1422349180c446db6c1768b3025253c (127.22.69.61:41945)
I20260810 01:57:42.699025 25189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "757bb3f336084510845ed58df264c721" candidate_term: 5 candidate_status { last_received { term: 4 index: 35 } } ignore_live_leader: false dest_uuid: "b1422349180c446db6c1768b3025253c"
I20260810 01:57:42.699268 25189 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 4 FOLLOWER]: Advancing to term 5
W20260810 01:57:42.699810 25346 leader_election.cc:336] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311): Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111)
I20260810 01:57:42.703250 25189 raft_consensus.cc:2468] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 757bb3f336084510845ed58df264c721 in term 5.
I20260810 01:57:42.703692 25344 leader_election.cc:304] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c; no voters: 919313c0dee540a690f46dbc8208d20b
I20260810 01:57:42.703981 25464 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 5 FOLLOWER]: Leader election won for term 5
I20260810 01:57:42.704432 25464 raft_consensus.cc:697] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 5 LEADER]: Becoming Leader. State: Replica: 757bb3f336084510845ed58df264c721, State: Running, Role: LEADER
I20260810 01:57:42.704874 25464 consensus_queue.cc:237] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 4.35, Last appended by leader: 35, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:42.707464 25470 sys_catalog.cc:455] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 757bb3f336084510845ed58df264c721. Latest consensus state: current_term: 5 leader_uuid: "757bb3f336084510845ed58df264c721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:42.707681 25470 sys_catalog.cc:458] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [sys.catalog]: This master's current role is: LEADER
I20260810 01:57:42.708355 25472 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260810 01:57:42.710855 25472 catalog_manager.cc:679] Loaded metadata for table default.table_72b6f93a124042e4a090dc194134c75a [id=017756566b72415aa33842f8df2b3e91]
I20260810 01:57:42.711407 25472 catalog_manager.cc:679] Loaded metadata for table default.table_73d521a2c5504815ad6d7cc3dd0bf818 [id=2f1832807601444a88d61b8396a67db9]
I20260810 01:57:42.711777 25472 catalog_manager.cc:679] Loaded metadata for table default.table_c8ad906be0f74210912a57adead21255 [id=4da3c920ea6a48588c9f0c91da00fb69]
I20260810 01:57:42.712131 25472 catalog_manager.cc:679] Loaded metadata for table default.table_983bf2e604df4fdbac75c5aca292f736 [id=552771dde220474a9be9abeefd3a0472]
I20260810 01:57:42.712463 25472 catalog_manager.cc:679] Loaded metadata for table default.table_22726fd1a11f409f9f5287ab16d8e551 [id=5edf9cb336414ae3bd41e9f5ca41fcca]
I20260810 01:57:42.712802 25472 catalog_manager.cc:679] Loaded metadata for table default.table_0ba2046f969346f68f1bde7cf6c9f273 [id=630b03060b7646118927a18573c0920c]
I20260810 01:57:42.713140 25472 catalog_manager.cc:679] Loaded metadata for table default.table_510ae1b5fcc74db788ffb64925cad250 [id=7d240d10089c47eaa014d59c4ac7f0b4]
I20260810 01:57:42.713485 25472 catalog_manager.cc:679] Loaded metadata for table default.table_379ea52611ef49d591a022addb7afc0c [id=87ba140e9080402aa494198d6f212f05]
I20260810 01:57:42.713811 25472 catalog_manager.cc:679] Loaded metadata for table default.table_57697192c9bf44969c2b200e94865a3e [id=ef2e836fffc64aa6a8ddc0425f39e317]
I20260810 01:57:42.715531 25472 tablet_loader.cc:96] loaded metadata for tablet 09ed68395e2d44a5b51a22abe2203406 (table default.table_73d521a2c5504815ad6d7cc3dd0bf818 [id=2f1832807601444a88d61b8396a67db9])
I20260810 01:57:42.716087 25472 tablet_loader.cc:96] loaded metadata for tablet 16db112abfb34d4a9f2ecbee98c39d8b (table default.table_72b6f93a124042e4a090dc194134c75a [id=017756566b72415aa33842f8df2b3e91])
I20260810 01:57:42.716574 25472 tablet_loader.cc:96] loaded metadata for tablet 1c3db8429f9f4cf5b29d3d1a6cc0651c (table default.table_379ea52611ef49d591a022addb7afc0c [id=87ba140e9080402aa494198d6f212f05])
I20260810 01:57:42.716872 25472 tablet_loader.cc:96] loaded metadata for tablet 2230d919953e49208659450a024c6a62 (table default.table_0ba2046f969346f68f1bde7cf6c9f273 [id=630b03060b7646118927a18573c0920c])
I20260810 01:57:42.717192 25472 tablet_loader.cc:96] loaded metadata for tablet 4579a90be0b6409781fd9e470324192c (table default.table_22726fd1a11f409f9f5287ab16d8e551 [id=5edf9cb336414ae3bd41e9f5ca41fcca])
I20260810 01:57:42.717453 25472 tablet_loader.cc:96] loaded metadata for tablet 5c49a207b37e40579d34002c3fb57afb (table default.table_c8ad906be0f74210912a57adead21255 [id=4da3c920ea6a48588c9f0c91da00fb69])
I20260810 01:57:42.717741 25472 tablet_loader.cc:96] loaded metadata for tablet 89d9556cc526444299c786de606bb5b1 (table default.table_510ae1b5fcc74db788ffb64925cad250 [id=7d240d10089c47eaa014d59c4ac7f0b4])
I20260810 01:57:42.718047 25472 tablet_loader.cc:96] loaded metadata for tablet a48e9f780b4e49abafaadbd78ab8711a (table default.table_57697192c9bf44969c2b200e94865a3e [id=ef2e836fffc64aa6a8ddc0425f39e317])
I20260810 01:57:42.718623 25472 tablet_loader.cc:96] loaded metadata for tablet e6ef3afe40a847dbb08792578d52401d (table default.table_983bf2e604df4fdbac75c5aca292f736 [id=552771dde220474a9be9abeefd3a0472])
I20260810 01:57:42.718874 25472 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260810 01:57:42.720077 25472 catalog_manager.cc:1295] Loaded cluster ID: dfa48c56b63b4d0cbf3d6124921e8a17
I20260810 01:57:42.720191 25472 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260810 01:57:42.721665 25472 catalog_manager.cc:1540] Loading token signing keys...
I20260810 01:57:42.722498 25472 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721: Loaded TSK: 0
I20260810 01:57:42.723405 25472 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260810 01:57:42.724383 25472 catalog_manager.cc:1563] Loading latest processed Hive Metastore notification log event ID...
I20260810 01:57:42.725334 25472 catalog_manager.cc:5941] Last processed Hive Metastore notification event ID: -1
I20260810 01:57:42.809894 25361 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59504:
table { table_name: "default.Table_510ae1b5fcc74db788ffb64925cad250" } modify_external_catalogs: true
I20260810 01:57:42.810202 25358 master_service.cc:960] ReplaceTablet: received request to replace tablet 89d9556cc526444299c786de606bb5b1 from {username='slave'} at 127.0.0.1:59504
I20260810 01:57:42.810302 25361 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59504:
table { table_name: "default.Table_510ae1b5fcc74db788ffb64925cad250" } modify_external_catalogs: true
I20260810 01:57:42.809919 25360 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59504:
name: "default.Table_7e5efb367a02411aac568a75dd39b274"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:42.812218 25359 master_service.cc:960] ReplaceTablet: received request to replace tablet 16db112abfb34d4a9f2ecbee98c39d8b from {username='slave'} at 127.0.0.1:59504
I20260810 01:57:42.812341 25353 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59504:
name: "default.Table_70288f170151490fb6ba63a3bf29f01e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:42.811980 25352 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:59504:
table { table_name: "default.Table_73d521a2c5504815ad6d7cc3dd0bf818" } new_table_name: "default.Table_f5851b3fa264482388e4e50f34d4e37c" modify_external_catalogs: true
I20260810 01:57:42.815441 25356 master_service.cc:960] ReplaceTablet: received request to replace tablet b24af70442e44ca4800206b1b5bd7715 from {username='slave'} at 127.0.0.1:59504
I20260810 01:57:42.815943 25355 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59504:
name: "default.Table_bb2d4e224bd14be09d60bce7977d4c19"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:42.816650 25354 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59504:
name: "default.Table_1bd23f5807da48238cbddc7597b8ab50"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:42.816854 25357 master_service.cc:960] ReplaceTablet: received request to replace tablet 09ed68395e2d44a5b51a22abe2203406 from {username='slave'} at 127.0.0.1:59504
I20260810 01:57:42.819698 25356 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:59504:
table { table_name: "default.Table_c8ad906be0f74210912a57adead21255" } new_table_name: "default.Table_e2cb378ea960473c833b1d18fbe32f91" modify_external_catalogs: true
F20260810 01:57:42.820386 25443 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 10 received events: NotificationEvent(eventId=1, eventTime=1786327049, eventType=CREATE_TABLE, dbName=default, tableName=table_4fb63c687c11493ca7c0b76735f137e9, message={"server":"","servicePrincipal":"","db":"default","table":"table_4fb63c687c11493ca7c0b76735f137e9","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_4fb63c687c11493ca7c0b76735f137e9\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1786327049},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/warehouse/table_4fb63c687c11493ca7c0b76735f137e9\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_4fb63c687c11493ca7c0b76735f137e9\",\"kudu.table_id\":\"cb924727e6014a2a9dd271f962df3807\",\"transient_lastDdlTime\":\"1786327049\",\"comment\":\"\",\"kudu.master_addresses\":\"127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"dfa48c56b63b4d0cbf3d6124921e8a17\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1786327049,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1786327049, eventType=CREATE_TABLE, dbName=default, tableName=table_c8ad906be0f74210912a57adead21255, message={"server":"","servicePrincipal":"","db":"default","table":"table_c8ad906be0f74210912a57adead21255","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_c8ad906be0f74210912a57adead21255\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1786327049},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/warehouse/table_c8ad906be0f74210912a57adead21255\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_c8ad906be0f74210912a57adead21255\",\"kudu.table_id\":\"4da3c920ea6a48588c9f0c91da00fb69\",\"transient_lastDdlTime\":\"1786327049\",\"comment\":\"\",\"kudu.master_addresses\":\"127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"dfa48c56b63b4d0cbf3d6124921e8a17\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1786327049,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
@ 0x7f07f1d86d5d google::LogMessage::Fail() at ??:0
I20260810 01:57:42.824186 25189 raft_consensus.cc:1275] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 5 FOLLOWER]: Refusing update from remote peer 757bb3f336084510845ed58df264c721: Log matching property violated. Preceding OpId in replica: term: 4 index: 35. Preceding OpId from leader: term: 5 index: 37. (index mismatch)
W20260810 01:57:42.824235 25346 consensus_peers.cc:597] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 -> Peer 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311): Couldn't send request to peer 919313c0dee540a690f46dbc8208d20b. Status: Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
@ 0x7f07f1d8ab23 google::LogMessage::SendToLog() at ??:0
I20260810 01:57:42.825089 25470 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
@ 0x7f07f1d8675c google::LogMessage::Flush() at ??:0
@ 0x7f07f1d869a1 google::LogMessage::~LogMessage() at ??:0
I20260810 01:57:42.836527 25478 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 757bb3f336084510845ed58df264c721. Latest consensus state: current_term: 5 leader_uuid: "757bb3f336084510845ed58df264c721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:42.836874 25478 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: This master's current role is: FOLLOWER
@ 0x7f0808a4ff64 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260810 01:57:42.851866 25359 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721: Sending DeleteTablet for 3 replicas of tablet 16db112abfb34d4a9f2ecbee98c39d8b
I20260810 01:57:42.851856 25478 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "757bb3f336084510845ed58df264c721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:42.852041 25478 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:57:42.852860 25470 sys_catalog.cc:455] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 5 leader_uuid: "757bb3f336084510845ed58df264c721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:42.853124 25470 sys_catalog.cc:458] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [sys.catalog]: This master's current role is: LEADER
I20260810 01:57:42.853582 25357 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721: Sending DeleteTablet for 3 replicas of tablet 09ed68395e2d44a5b51a22abe2203406
I20260810 01:57:42.853778 25358 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721: Sending DeleteTablet for 3 replicas of tablet 89d9556cc526444299c786de606bb5b1
I20260810 01:57:42.854574 25357 catalog_manager.cc:6807] ReplaceTablet: tablet 09ed68395e2d44a5b51a22abe2203406 deleted and replaced by tablet cad2404759604bfea7c273e1aa40539a
I20260810 01:57:42.854655 25358 catalog_manager.cc:6807] ReplaceTablet: tablet 89d9556cc526444299c786de606bb5b1 deleted and replaced by tablet 4514cc930ba44e0f92a438f8ae0ba6fc
I20260810 01:57:42.854936 25357 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:59504:
table { table_name: "default.Table_72b6f93a124042e4a090dc194134c75a" } new_table_name: "default.Table_b8900cd2c2574605ade34fc7ccc46f94" modify_external_catalogs: true
I20260810 01:57:42.855082 25358 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59504:
name: "default.Table_73822ba0bac443d19cc7c66f6f547bda"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:42.859368 25359 catalog_manager.cc:6807] ReplaceTablet: tablet 16db112abfb34d4a9f2ecbee98c39d8b deleted and replaced by tablet 6ca335f1e5964f8490a49ba4c4fb4c72
I20260810 01:57:42.859790 25359 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:59504:
table { table_name: "default.Table_379ea52611ef49d591a022addb7afc0c" } new_table_name: "default.Table_09a7f5150ded428991a5d464657e85ae" modify_external_catalogs: true
@ 0x7f0808a4e811 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f0808a5648c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7f0808a56309 std::_Function_handler<>::_M_invoke() at ??:0
@ 0xaaf86d std::function<>::operator()() at ??:?
@ 0x7f07f4aaedd9 kudu::Thread::SuperviseThread() at ??:0
I20260810 01:57:42.881560 23746 tablet_service.cc:1558] Processing DeleteTablet for tablet 16db112abfb34d4a9f2ecbee98c39d8b with delete_type TABLET_DATA_DELETED (replaced by tablet 6ca335f1e5964f8490a49ba4c4fb4c72) from {username='slave'} at 127.0.0.1:46844
I20260810 01:57:42.884045 23747 tablet_service.cc:1558] Processing DeleteTablet for tablet 09ed68395e2d44a5b51a22abe2203406 with delete_type TABLET_DATA_DELETED (replaced by tablet cad2404759604bfea7c273e1aa40539a) from {username='slave'} at 127.0.0.1:46844
@ 0x7f07f817e6db start_thread at ??:0
I20260810 01:57:42.884608 23747 tablet_service.cc:1558] Processing DeleteTablet for tablet 89d9556cc526444299c786de606bb5b1 with delete_type TABLET_DATA_DELETED (replaced by tablet 4514cc930ba44e0f92a438f8ae0ba6fc) from {username='slave'} at 127.0.0.1:46844
I20260810 01:57:42.885272 24187 tablet_service.cc:1558] Processing DeleteTablet for tablet 89d9556cc526444299c786de606bb5b1 with delete_type TABLET_DATA_DELETED (replaced by tablet 4514cc930ba44e0f92a438f8ae0ba6fc) from {username='slave'} at 127.0.0.1:37366
I20260810 01:57:42.885371 23747 tablet_service.cc:1511] Processing CreateTablet for tablet 4514cc930ba44e0f92a438f8ae0ba6fc (DEFAULT_TABLE table=default.table_510ae1b5fcc74db788ffb64925cad250 [id=7d240d10089c47eaa014d59c4ac7f0b4]), partition=RANGE (key) PARTITION UNBOUNDED
@ 0x7f07f0c3561f clone at ??:0
I20260810 01:57:42.885860 23747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4514cc930ba44e0f92a438f8ae0ba6fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:42.886191 24191 tablet_service.cc:1558] Processing DeleteTablet for tablet 09ed68395e2d44a5b51a22abe2203406 with delete_type TABLET_DATA_DELETED (replaced by tablet cad2404759604bfea7c273e1aa40539a) from {username='slave'} at 127.0.0.1:37366
I20260810 01:57:42.886484 24186 tablet_service.cc:1558] Processing DeleteTablet for tablet 16db112abfb34d4a9f2ecbee98c39d8b with delete_type TABLET_DATA_DELETED (replaced by tablet 6ca335f1e5964f8490a49ba4c4fb4c72) from {username='slave'} at 127.0.0.1:37366
I20260810 01:57:42.886868 24185 tablet_service.cc:1511] Processing CreateTablet for tablet 6ca335f1e5964f8490a49ba4c4fb4c72 (DEFAULT_TABLE table=default.table_72b6f93a124042e4a090dc194134c75a [id=017756566b72415aa33842f8df2b3e91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:42.887008 24192 tablet_service.cc:1511] Processing CreateTablet for tablet 4514cc930ba44e0f92a438f8ae0ba6fc (DEFAULT_TABLE table=default.table_510ae1b5fcc74db788ffb64925cad250 [id=7d240d10089c47eaa014d59c4ac7f0b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:42.887398 24185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ca335f1e5964f8490a49ba4c4fb4c72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:42.887677 24192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4514cc930ba44e0f92a438f8ae0ba6fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:42.895671 24186 tablet_service.cc:1511] Processing CreateTablet for tablet 5c49a207b37e40579d34002c3fb57afb (DEFAULT_TABLE table=default.table_c8ad906be0f74210912a57adead21255 [id=4da3c920ea6a48588c9f0c91da00fb69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:42.896114 24186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c49a207b37e40579d34002c3fb57afb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:42.899497 24191 tablet_service.cc:1511] Processing CreateTablet for tablet cad2404759604bfea7c273e1aa40539a (DEFAULT_TABLE table=default.table_73d521a2c5504815ad6d7cc3dd0bf818 [id=2f1832807601444a88d61b8396a67db9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:42.899997 24191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cad2404759604bfea7c273e1aa40539a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:42.901041 25512 tablet_replica.cc:333] T 89d9556cc526444299c786de606bb5b1 P 498d352dbea9405199a121e84a4ab926: stopping tablet replica
I20260810 01:57:42.901266 25512 raft_consensus.cc:2243] T 89d9556cc526444299c786de606bb5b1 P 498d352dbea9405199a121e84a4ab926 [term 1 LEADER]: Raft consensus shutting down.
I20260810 01:57:42.901925 25512 raft_consensus.cc:2272] T 89d9556cc526444299c786de606bb5b1 P 498d352dbea9405199a121e84a4ab926 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:57:42.902570 25512 ts_tablet_manager.cc:1916] T 89d9556cc526444299c786de606bb5b1 P 498d352dbea9405199a121e84a4ab926: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:57:42.903391 25513 tablet_bootstrap.cc:492] T 4514cc930ba44e0f92a438f8ae0ba6fc P 50d44445234b4931aaa28141cdaa67ab: Bootstrap starting.
I20260810 01:57:42.908974 25510 tablet_replica.cc:333] T 16db112abfb34d4a9f2ecbee98c39d8b P 50d44445234b4931aaa28141cdaa67ab: stopping tablet replica
I20260810 01:57:42.910414 25513 tablet_bootstrap.cc:654] T 4514cc930ba44e0f92a438f8ae0ba6fc P 50d44445234b4931aaa28141cdaa67ab: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:42.911749 25510 raft_consensus.cc:2243] T 16db112abfb34d4a9f2ecbee98c39d8b P 50d44445234b4931aaa28141cdaa67ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 01:57:42.912029 25510 raft_consensus.cc:2272] T 16db112abfb34d4a9f2ecbee98c39d8b P 50d44445234b4931aaa28141cdaa67ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:57:42.912792 25510 ts_tablet_manager.cc:1916] T 16db112abfb34d4a9f2ecbee98c39d8b P 50d44445234b4931aaa28141cdaa67ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:57:42.917657 25514 tablet_bootstrap.cc:492] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926: Bootstrap starting.
I20260810 01:57:42.918632 25513 tablet_bootstrap.cc:492] T 4514cc930ba44e0f92a438f8ae0ba6fc P 50d44445234b4931aaa28141cdaa67ab: No bootstrap required, opened a new log
I20260810 01:57:42.918781 25513 ts_tablet_manager.cc:1403] T 4514cc930ba44e0f92a438f8ae0ba6fc P 50d44445234b4931aaa28141cdaa67ab: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.000s
I20260810 01:57:42.919636 25514 tablet_bootstrap.cc:654] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:42.919660 25513 raft_consensus.cc:359] T 4514cc930ba44e0f92a438f8ae0ba6fc P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } }
I20260810 01:57:42.919876 25513 raft_consensus.cc:385] T 4514cc930ba44e0f92a438f8ae0ba6fc P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:42.919970 25513 raft_consensus.cc:740] T 4514cc930ba44e0f92a438f8ae0ba6fc P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50d44445234b4931aaa28141cdaa67ab, State: Initialized, Role: FOLLOWER
I20260810 01:57:42.920271 25513 consensus_queue.cc:260] T 4514cc930ba44e0f92a438f8ae0ba6fc P 50d44445234b4931aaa28141cdaa67ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } }
I20260810 01:57:42.920856 25513 ts_tablet_manager.cc:1434] T 4514cc930ba44e0f92a438f8ae0ba6fc P 50d44445234b4931aaa28141cdaa67ab: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 01:57:42.923050 25512 ts_tablet_manager.cc:1929] T 89d9556cc526444299c786de606bb5b1 P 498d352dbea9405199a121e84a4ab926: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 01:57:42.923192 25512 log.cc:1199] T 89d9556cc526444299c786de606bb5b1 P 498d352dbea9405199a121e84a4ab926: Deleting WAL directory at /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-2/wal/wals/89d9556cc526444299c786de606bb5b1
I20260810 01:57:42.923673 25512 ts_tablet_manager.cc:1950] T 89d9556cc526444299c786de606bb5b1 P 498d352dbea9405199a121e84a4ab926: Deleting consensus metadata
I20260810 01:57:42.925468 25514 tablet_bootstrap.cc:492] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926: No bootstrap required, opened a new log
I20260810 01:57:42.925616 25514 ts_tablet_manager.cc:1403] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.002s
I20260810 01:57:42.926362 25514 raft_consensus.cc:359] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:42.926555 25514 raft_consensus.cc:385] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:42.926647 25514 raft_consensus.cc:740] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 498d352dbea9405199a121e84a4ab926, State: Initialized, Role: FOLLOWER
I20260810 01:57:42.926911 25514 consensus_queue.cc:260] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:42.927456 25514 ts_tablet_manager.cc:1434] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 01:57:42.927711 24298 heartbeater.cc:499] Master 127.22.69.60:36917 was elected leader, sending a full tablet report...
I20260810 01:57:42.928301 25510 ts_tablet_manager.cc:1929] T 16db112abfb34d4a9f2ecbee98c39d8b P 50d44445234b4931aaa28141cdaa67ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 01:57:42.928387 25510 log.cc:1199] T 16db112abfb34d4a9f2ecbee98c39d8b P 50d44445234b4931aaa28141cdaa67ab: Deleting WAL directory at /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-0/wal/wals/16db112abfb34d4a9f2ecbee98c39d8b
I20260810 01:57:42.928826 25510 ts_tablet_manager.cc:1950] T 16db112abfb34d4a9f2ecbee98c39d8b P 50d44445234b4931aaa28141cdaa67ab: Deleting consensus metadata
I20260810 01:57:42.929778 25510 tablet_replica.cc:333] T 09ed68395e2d44a5b51a22abe2203406 P 50d44445234b4931aaa28141cdaa67ab: stopping tablet replica
I20260810 01:57:42.929983 25510 raft_consensus.cc:2243] T 09ed68395e2d44a5b51a22abe2203406 P 50d44445234b4931aaa28141cdaa67ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 01:57:42.930181 25510 raft_consensus.cc:2272] T 09ed68395e2d44a5b51a22abe2203406 P 50d44445234b4931aaa28141cdaa67ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:57:42.930675 25510 ts_tablet_manager.cc:1916] T 09ed68395e2d44a5b51a22abe2203406 P 50d44445234b4931aaa28141cdaa67ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:57:42.931347 25514 tablet_bootstrap.cc:492] T 4514cc930ba44e0f92a438f8ae0ba6fc P 498d352dbea9405199a121e84a4ab926: Bootstrap starting.
I20260810 01:57:42.933059 25514 tablet_bootstrap.cc:654] T 4514cc930ba44e0f92a438f8ae0ba6fc P 498d352dbea9405199a121e84a4ab926: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:42.933997 25512 tablet_replica.cc:333] T 09ed68395e2d44a5b51a22abe2203406 P 498d352dbea9405199a121e84a4ab926: stopping tablet replica
I20260810 01:57:42.934219 25512 raft_consensus.cc:2243] T 09ed68395e2d44a5b51a22abe2203406 P 498d352dbea9405199a121e84a4ab926 [term 1 LEADER]: Raft consensus shutting down.
I20260810 01:57:42.934547 25512 raft_consensus.cc:2272] T 09ed68395e2d44a5b51a22abe2203406 P 498d352dbea9405199a121e84a4ab926 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:57:42.935106 25512 ts_tablet_manager.cc:1916] T 09ed68395e2d44a5b51a22abe2203406 P 498d352dbea9405199a121e84a4ab926: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:57:42.935997 25510 ts_tablet_manager.cc:1929] T 09ed68395e2d44a5b51a22abe2203406 P 50d44445234b4931aaa28141cdaa67ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 01:57:42.936103 25510 log.cc:1199] T 09ed68395e2d44a5b51a22abe2203406 P 50d44445234b4931aaa28141cdaa67ab: Deleting WAL directory at /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-0/wal/wals/09ed68395e2d44a5b51a22abe2203406
I20260810 01:57:42.936396 25510 ts_tablet_manager.cc:1950] T 09ed68395e2d44a5b51a22abe2203406 P 50d44445234b4931aaa28141cdaa67ab: Deleting consensus metadata
I20260810 01:57:42.937129 25510 tablet_replica.cc:333] T 89d9556cc526444299c786de606bb5b1 P 50d44445234b4931aaa28141cdaa67ab: stopping tablet replica
I20260810 01:57:42.937300 25510 raft_consensus.cc:2243] T 89d9556cc526444299c786de606bb5b1 P 50d44445234b4931aaa28141cdaa67ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 01:57:42.937484 25510 raft_consensus.cc:2272] T 89d9556cc526444299c786de606bb5b1 P 50d44445234b4931aaa28141cdaa67ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:57:42.938155 25510 ts_tablet_manager.cc:1916] T 89d9556cc526444299c786de606bb5b1 P 50d44445234b4931aaa28141cdaa67ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:57:42.941159 25514 tablet_bootstrap.cc:492] T 4514cc930ba44e0f92a438f8ae0ba6fc P 498d352dbea9405199a121e84a4ab926: No bootstrap required, opened a new log
I20260810 01:57:42.941314 25514 ts_tablet_manager.cc:1403] T 4514cc930ba44e0f92a438f8ae0ba6fc P 498d352dbea9405199a121e84a4ab926: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.001s
I20260810 01:57:42.942130 25514 raft_consensus.cc:359] T 4514cc930ba44e0f92a438f8ae0ba6fc P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } }
I20260810 01:57:42.942333 25514 raft_consensus.cc:385] T 4514cc930ba44e0f92a438f8ae0ba6fc P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:42.942443 25514 raft_consensus.cc:740] T 4514cc930ba44e0f92a438f8ae0ba6fc P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 498d352dbea9405199a121e84a4ab926, State: Initialized, Role: FOLLOWER
I20260810 01:57:42.942759 25514 consensus_queue.cc:260] T 4514cc930ba44e0f92a438f8ae0ba6fc P 498d352dbea9405199a121e84a4ab926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } }
I20260810 01:57:42.943301 25514 ts_tablet_manager.cc:1434] T 4514cc930ba44e0f92a438f8ae0ba6fc P 498d352dbea9405199a121e84a4ab926: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260810 01:57:42.943686 25514 tablet_bootstrap.cc:492] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926: Bootstrap starting.
I20260810 01:57:42.944411 25510 ts_tablet_manager.cc:1929] T 89d9556cc526444299c786de606bb5b1 P 50d44445234b4931aaa28141cdaa67ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 01:57:42.944523 25510 log.cc:1199] T 89d9556cc526444299c786de606bb5b1 P 50d44445234b4931aaa28141cdaa67ab: Deleting WAL directory at /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-0/wal/wals/89d9556cc526444299c786de606bb5b1
I20260810 01:57:42.944911 25510 ts_tablet_manager.cc:1950] T 89d9556cc526444299c786de606bb5b1 P 50d44445234b4931aaa28141cdaa67ab: Deleting consensus metadata
I20260810 01:57:42.945505 25514 tablet_bootstrap.cc:654] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:42.948388 25512 ts_tablet_manager.cc:1929] T 09ed68395e2d44a5b51a22abe2203406 P 498d352dbea9405199a121e84a4ab926: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 01:57:42.948539 25512 log.cc:1199] T 09ed68395e2d44a5b51a22abe2203406 P 498d352dbea9405199a121e84a4ab926: Deleting WAL directory at /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-2/wal/wals/09ed68395e2d44a5b51a22abe2203406
I20260810 01:57:42.948741 25514 tablet_bootstrap.cc:492] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926: No bootstrap required, opened a new log
I20260810 01:57:42.948882 25514 ts_tablet_manager.cc:1403] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260810 01:57:42.949033 25512 ts_tablet_manager.cc:1950] T 09ed68395e2d44a5b51a22abe2203406 P 498d352dbea9405199a121e84a4ab926: Deleting consensus metadata
I20260810 01:57:42.949671 25514 raft_consensus.cc:359] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:42.949875 25514 raft_consensus.cc:385] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:42.949970 25514 raft_consensus.cc:740] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 498d352dbea9405199a121e84a4ab926, State: Initialized, Role: FOLLOWER
I20260810 01:57:42.950242 25512 tablet_replica.cc:333] T 16db112abfb34d4a9f2ecbee98c39d8b P 498d352dbea9405199a121e84a4ab926: stopping tablet replica
I20260810 01:57:42.950189 25514 consensus_queue.cc:260] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:42.950444 25512 raft_consensus.cc:2243] T 16db112abfb34d4a9f2ecbee98c39d8b P 498d352dbea9405199a121e84a4ab926 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 01:57:42.950637 25512 raft_consensus.cc:2272] T 16db112abfb34d4a9f2ecbee98c39d8b P 498d352dbea9405199a121e84a4ab926 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:57:42.950737 25514 ts_tablet_manager.cc:1434] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260810 01:57:42.951146 25514 tablet_bootstrap.cc:492] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926: Bootstrap starting.
I20260810 01:57:42.951200 25512 ts_tablet_manager.cc:1916] T 16db112abfb34d4a9f2ecbee98c39d8b P 498d352dbea9405199a121e84a4ab926: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:57:42.954087 25514 tablet_bootstrap.cc:654] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:42.956802 25514 tablet_bootstrap.cc:492] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926: No bootstrap required, opened a new log
I20260810 01:57:42.956976 25514 ts_tablet_manager.cc:1403] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260810 01:57:42.957698 25514 raft_consensus.cc:359] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:42.957894 25514 raft_consensus.cc:385] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:42.957999 25514 raft_consensus.cc:740] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 498d352dbea9405199a121e84a4ab926, State: Initialized, Role: FOLLOWER
I20260810 01:57:42.958278 25514 consensus_queue.cc:260] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:42.958808 25514 ts_tablet_manager.cc:1434] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260810 01:57:42.959597 25512 ts_tablet_manager.cc:1929] T 16db112abfb34d4a9f2ecbee98c39d8b P 498d352dbea9405199a121e84a4ab926: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 01:57:42.959756 25512 log.cc:1199] T 16db112abfb34d4a9f2ecbee98c39d8b P 498d352dbea9405199a121e84a4ab926: Deleting WAL directory at /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-2/wal/wals/16db112abfb34d4a9f2ecbee98c39d8b
I20260810 01:57:42.960242 25512 ts_tablet_manager.cc:1950] T 16db112abfb34d4a9f2ecbee98c39d8b P 498d352dbea9405199a121e84a4ab926: Deleting consensus metadata
W20260810 01:57:43.026798 23889 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Couldn't send request to peer 50d44445234b4931aaa28141cdaa67ab. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 1: this message will repeat every 5th retry.
W20260810 01:57:43.102109 23684 connection.cc:573] server connection from 127.0.0.1:46844 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 01:57:43.102093 24114 connection.cc:573] server connection from 127.0.0.1:37366 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 01:57:43.102109 24111 connection.cc:573] client connection to 127.22.69.60:36917 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 01:57:43.102705 23681 connection.cc:573] client connection to 127.22.69.60:36917 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 01:57:43.102757 24317 connection.cc:573] client connection to 127.22.69.60:36917 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 01:57:43.103478 24317 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.22.69.60:36917)
W20260810 01:57:43.104063 24317 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.22.69.60:36917)
W20260810 01:57:43.104285 24317 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.22.69.60:36917)
W20260810 01:57:43.104713 23890 connection.cc:573] client connection to 127.22.69.60:36917 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 01:57:43.105342 24317 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.22.69.60:36917)
W20260810 01:57:43.154398 23892 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 1: this message will repeat every 5th retry.
I20260810 01:57:43.211788 24564 raft_consensus.cc:493] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:57:43.211999 24564 raft_consensus.cc:515] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:43.212771 24564 leader_election.cc:290] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:57:43.213963 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
W20260810 01:57:43.214326 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
I20260810 01:57:43.214432 24109 leader_election.cc:304] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:57:43.214654 24564 raft_consensus.cc:2749] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:57:43.261241 25473 raft_consensus.cc:493] T 4514cc930ba44e0f92a438f8ae0ba6fc P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:57:43.261423 25473 raft_consensus.cc:515] T 4514cc930ba44e0f92a438f8ae0ba6fc P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } }
I20260810 01:57:43.262066 25473 leader_election.cc:290] T 4514cc930ba44e0f92a438f8ae0ba6fc P 50d44445234b4931aaa28141cdaa67ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
I20260810 01:57:43.262563 24222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4514cc930ba44e0f92a438f8ae0ba6fc" candidate_uuid: "50d44445234b4931aaa28141cdaa67ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "498d352dbea9405199a121e84a4ab926" is_pre_election: true
I20260810 01:57:43.262773 24222 raft_consensus.cc:2468] T 4514cc930ba44e0f92a438f8ae0ba6fc P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50d44445234b4931aaa28141cdaa67ab in term 0.
W20260810 01:57:43.263204 23679 leader_election.cc:343] T 4514cc930ba44e0f92a438f8ae0ba6fc P 50d44445234b4931aaa28141cdaa67ab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 4514cc930ba44e0f92a438f8ae0ba6fc
I20260810 01:57:43.263489 23684 leader_election.cc:304] T 4514cc930ba44e0f92a438f8ae0ba6fc P 50d44445234b4931aaa28141cdaa67ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab; no voters: 48e33bc1e8e94d4887bb64214d0b6c36
I20260810 01:57:43.263760 25473 raft_consensus.cc:2804] T 4514cc930ba44e0f92a438f8ae0ba6fc P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 01:57:43.263882 25473 raft_consensus.cc:493] T 4514cc930ba44e0f92a438f8ae0ba6fc P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 01:57:43.263975 25473 raft_consensus.cc:3060] T 4514cc930ba44e0f92a438f8ae0ba6fc P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:43.266538 25473 raft_consensus.cc:515] T 4514cc930ba44e0f92a438f8ae0ba6fc P 50d44445234b4931aaa28141cdaa67ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } }
I20260810 01:57:43.267334 25473 leader_election.cc:290] T 4514cc930ba44e0f92a438f8ae0ba6fc P 50d44445234b4931aaa28141cdaa67ab [CANDIDATE]: Term 1 election: Requested vote from peers 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
I20260810 01:57:43.267813 24222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4514cc930ba44e0f92a438f8ae0ba6fc" candidate_uuid: "50d44445234b4931aaa28141cdaa67ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "498d352dbea9405199a121e84a4ab926"
I20260810 01:57:43.268010 24222 raft_consensus.cc:3060] T 4514cc930ba44e0f92a438f8ae0ba6fc P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Advancing to term 1
W20260810 01:57:43.268149 23679 leader_election.cc:343] T 4514cc930ba44e0f92a438f8ae0ba6fc P 50d44445234b4931aaa28141cdaa67ab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 4514cc930ba44e0f92a438f8ae0ba6fc
I20260810 01:57:43.270047 24222 raft_consensus.cc:2468] T 4514cc930ba44e0f92a438f8ae0ba6fc P 498d352dbea9405199a121e84a4ab926 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50d44445234b4931aaa28141cdaa67ab in term 1.
I20260810 01:57:43.270416 23684 leader_election.cc:304] T 4514cc930ba44e0f92a438f8ae0ba6fc P 50d44445234b4931aaa28141cdaa67ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab; no voters: 48e33bc1e8e94d4887bb64214d0b6c36
I20260810 01:57:43.270678 25473 raft_consensus.cc:2804] T 4514cc930ba44e0f92a438f8ae0ba6fc P 50d44445234b4931aaa28141cdaa67ab [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:57:43.271102 25473 raft_consensus.cc:697] T 4514cc930ba44e0f92a438f8ae0ba6fc P 50d44445234b4931aaa28141cdaa67ab [term 1 LEADER]: Becoming Leader. State: Replica: 50d44445234b4931aaa28141cdaa67ab, State: Running, Role: LEADER
I20260810 01:57:43.271466 25473 consensus_queue.cc:237] T 4514cc930ba44e0f92a438f8ae0ba6fc P 50d44445234b4931aaa28141cdaa67ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } }
I20260810 01:57:43.370908 25540 raft_consensus.cc:493] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 757bb3f336084510845ed58df264c721)
I20260810 01:57:43.371146 25540 raft_consensus.cc:515] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:43.371796 25540 leader_election.cc:290] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:57:43.373631 25145 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311): Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111)
W20260810 01:57:43.373829 25142 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:57:43.373934 25142 leader_election.cc:304] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1422349180c446db6c1768b3025253c; no voters: 757bb3f336084510845ed58df264c721, 919313c0dee540a690f46dbc8208d20b
I20260810 01:57:43.374132 25540 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260810 01:57:43.508009 25117 master_runner.cc:187] Error getting master registration for 127.22.69.62:35311: Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111),
I20260810 01:57:43.508209 25117 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 01:57:43.605023 24564 raft_consensus.cc:493] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:57:43.605250 24564 raft_consensus.cc:515] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:43.605960 24564 leader_election.cc:290] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:57:43.606762 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
W20260810 01:57:43.606977 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
I20260810 01:57:43.607127 24109 leader_election.cc:304] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:57:43.607405 24564 raft_consensus.cc:2749] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:57:43.635193 24564 raft_consensus.cc:493] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:57:43.635380 24564 raft_consensus.cc:515] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:43.635910 24564 leader_election.cc:290] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:57:43.636891 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
W20260810 01:57:43.637133 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
I20260810 01:57:43.637244 24109 leader_election.cc:304] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:57:43.637538 24564 raft_consensus.cc:2749] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:57:44.030967 25555 raft_consensus.cc:493] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 757bb3f336084510845ed58df264c721)
I20260810 01:57:44.031265 25555 raft_consensus.cc:515] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:44.031991 25555 leader_election.cc:290] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:57:44.032909 25142 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111) [suppressed 11 similar messages]
W20260810 01:57:44.034080 25145 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311): Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111)
W20260810 01:57:44.034255 25142 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:57:44.034428 25142 leader_election.cc:304] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1422349180c446db6c1768b3025253c; no voters: 757bb3f336084510845ed58df264c721, 919313c0dee540a690f46dbc8208d20b
I20260810 01:57:44.034715 25555 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260810 01:57:44.035104 25554 consensus_queue.cc:1048] T 4514cc930ba44e0f92a438f8ae0ba6fc P 50d44445234b4931aaa28141cdaa67ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 01:57:44.041196 23679 consensus_peers.cc:597] T 4514cc930ba44e0f92a438f8ae0ba6fc P 50d44445234b4931aaa28141cdaa67ab -> Peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Couldn't send request to peer 48e33bc1e8e94d4887bb64214d0b6c36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4514cc930ba44e0f92a438f8ae0ba6fc. This is attempt 1: this message will repeat every 5th retry.
I20260810 01:57:44.905826 25578 ts_tablet_manager.cc:933] T 4514cc930ba44e0f92a438f8ae0ba6fc P 48e33bc1e8e94d4887bb64214d0b6c36: Initiating tablet copy from peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491)
I20260810 01:57:44.906947 25578 tablet_copy_client.cc:323] T 4514cc930ba44e0f92a438f8ae0ba6fc P 48e33bc1e8e94d4887bb64214d0b6c36: tablet copy: Beginning tablet copy session from remote peer at address 127.22.69.1:45491
I20260810 01:57:44.908245 23810 tablet_copy_service.cc:140] P 50d44445234b4931aaa28141cdaa67ab: Received BeginTabletCopySession request for tablet 4514cc930ba44e0f92a438f8ae0ba6fc from peer 48e33bc1e8e94d4887bb64214d0b6c36 ({username='slave'} at 127.22.69.2:32907)
I20260810 01:57:44.908542 23810 tablet_copy_service.cc:161] P 50d44445234b4931aaa28141cdaa67ab: Beginning new tablet copy session on tablet 4514cc930ba44e0f92a438f8ae0ba6fc from peer 48e33bc1e8e94d4887bb64214d0b6c36 at {username='slave'} at 127.22.69.2:32907: session id = 48e33bc1e8e94d4887bb64214d0b6c36-4514cc930ba44e0f92a438f8ae0ba6fc
I20260810 01:57:44.911660 23810 tablet_copy_source_session.cc:215] T 4514cc930ba44e0f92a438f8ae0ba6fc P 50d44445234b4931aaa28141cdaa67ab: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 01:57:44.914186 25578 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4514cc930ba44e0f92a438f8ae0ba6fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:44.919027 25578 tablet_copy_client.cc:806] T 4514cc930ba44e0f92a438f8ae0ba6fc P 48e33bc1e8e94d4887bb64214d0b6c36: tablet copy: Starting download of 0 data blocks...
I20260810 01:57:44.919441 25578 tablet_copy_client.cc:670] T 4514cc930ba44e0f92a438f8ae0ba6fc P 48e33bc1e8e94d4887bb64214d0b6c36: tablet copy: Starting download of 1 WAL segments...
I20260810 01:57:44.922065 25578 tablet_copy_client.cc:538] T 4514cc930ba44e0f92a438f8ae0ba6fc P 48e33bc1e8e94d4887bb64214d0b6c36: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 01:57:44.925012 25578 tablet_bootstrap.cc:492] T 4514cc930ba44e0f92a438f8ae0ba6fc P 48e33bc1e8e94d4887bb64214d0b6c36: Bootstrap starting.
I20260810 01:57:44.930524 25578 tablet_bootstrap.cc:492] T 4514cc930ba44e0f92a438f8ae0ba6fc P 48e33bc1e8e94d4887bb64214d0b6c36: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 01:57:44.930959 25578 tablet_bootstrap.cc:492] T 4514cc930ba44e0f92a438f8ae0ba6fc P 48e33bc1e8e94d4887bb64214d0b6c36: Bootstrap complete.
I20260810 01:57:44.931268 25578 ts_tablet_manager.cc:1403] T 4514cc930ba44e0f92a438f8ae0ba6fc P 48e33bc1e8e94d4887bb64214d0b6c36: Time spent bootstrapping tablet: real 0.006s user 0.007s sys 0.001s
I20260810 01:57:44.931947 25578 raft_consensus.cc:359] T 4514cc930ba44e0f92a438f8ae0ba6fc P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } }
I20260810 01:57:44.932121 25578 raft_consensus.cc:740] T 4514cc930ba44e0f92a438f8ae0ba6fc P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48e33bc1e8e94d4887bb64214d0b6c36, State: Initialized, Role: FOLLOWER
I20260810 01:57:44.932410 25578 consensus_queue.cc:260] T 4514cc930ba44e0f92a438f8ae0ba6fc P 48e33bc1e8e94d4887bb64214d0b6c36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } }
I20260810 01:57:44.933594 25578 ts_tablet_manager.cc:1434] T 4514cc930ba44e0f92a438f8ae0ba6fc P 48e33bc1e8e94d4887bb64214d0b6c36: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 01:57:44.934842 23810 tablet_copy_service.cc:342] P 50d44445234b4931aaa28141cdaa67ab: Request end of tablet copy session 48e33bc1e8e94d4887bb64214d0b6c36-4514cc930ba44e0f92a438f8ae0ba6fc received from {username='slave'} at 127.22.69.2:32907
I20260810 01:57:44.935134 23810 tablet_copy_service.cc:434] P 50d44445234b4931aaa28141cdaa67ab: ending tablet copy session 48e33bc1e8e94d4887bb64214d0b6c36-4514cc930ba44e0f92a438f8ae0ba6fc on tablet 4514cc930ba44e0f92a438f8ae0ba6fc with peer 48e33bc1e8e94d4887bb64214d0b6c36
I20260810 01:57:45.112876 25593 raft_consensus.cc:493] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 757bb3f336084510845ed58df264c721)
I20260810 01:57:45.113119 25593 raft_consensus.cc:515] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:45.113790 25593 leader_election.cc:290] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:57:45.116060 25142 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
W20260810 01:57:45.116263 25145 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311): Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111)
I20260810 01:57:45.116374 25145 leader_election.cc:304] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1422349180c446db6c1768b3025253c; no voters: 757bb3f336084510845ed58df264c721, 919313c0dee540a690f46dbc8208d20b
I20260810 01:57:45.116590 25593 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260810 01:57:45.188138 22804 external_mini_cluster.cc:1372] Running /tmp/dist-test-task_qhVVJ/build/asan/bin/kudu
/tmp/dist-test-task_qhVVJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.69.60:36917
--webserver_interface=127.22.69.60
--webserver_port=44467
--builtin_ntp_servers=127.22.69.20:43209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917
--hive_metastore_uris=thrift://127.0.0.1:37513
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260810 01:57:45.249320 25592 raft_consensus.cc:493] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 01:57:45.249614 25592 raft_consensus.cc:515] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:45.250576 25592 leader_election.cc:290] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 01:57:45.252813 23892 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
W20260810 01:57:45.253080 23889 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
I20260810 01:57:45.253237 23889 leader_election.cc:304] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:57:45.253533 25017 raft_consensus.cc:2749] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 01:57:45.311851 25017 raft_consensus.cc:493] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 01:57:45.312099 25017 raft_consensus.cc:515] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:45.312778 25017 leader_election.cc:290] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 01:57:45.313798 23889 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
W20260810 01:57:45.313977 23892 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
I20260810 01:57:45.314128 23892 leader_election.cc:304] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:57:45.314546 25017 raft_consensus.cc:2749] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 01:57:45.525573 25602 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:57:45.525987 25602 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:57:45.526158 25602 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260810 01:57:45.534145 25602 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:57:45.534263 25602 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:57:45.534363 25602 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:57:45.534458 25602 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:57:45.544032 25602 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.69.20:43209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:37513
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.69.60:36917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.69.60
--webserver_port=44467
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 10 Aug 2026 01:44:04 UTC on e7f111948823
build id 13266
ASAN enabled
I20260810 01:57:45.546209 25602 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:57:45.548385 25602 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260810 01:57:45.557046 25622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:57:45.558180 25624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:57:45.559196 25628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:57:45.559655 25602 server_base.cc:1061] running on GCE node
I20260810 01:57:45.560438 25602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:57:45.561764 25602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:57:45.562989 25602 hybrid_clock.cc:648] HybridClock initialized: now 1786327065562904 us; error 74 us; skew 500 ppm
I20260810 01:57:45.563357 25602 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 01:57:45.565387 25602 webserver.cc:533] Webserver started at http://127.22.69.60:44467/ using document root <none> and password file <none>
I20260810 01:57:45.566202 25602 fs_manager.cc:362] Metadata directory not provided
I20260810 01:57:45.566352 25602 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:57:45.572009 25602 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 01:57:45.574743 25637 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:57:45.576090 25602 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 01:57:45.576221 25602 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/data,/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/wal
uuid: "757bb3f336084510845ed58df264c721"
format_stamp: "Formatted at 2026-08-10 01:57:26 on dist-test-slave-n326"
I20260810 01:57:45.576859 25602 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:57:45.595894 25602 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260810 01:57:45.596668 25602 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:57:45.596909 25602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:57:45.617524 25602 rpc_server.cc:307] RPC server started. Bound to: 127.22.69.60:36917
I20260810 01:57:45.617547 25713 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.22.69.60:36917 every 8 connection(s)
I20260810 01:57:45.619192 25602 server_base.cc:1193] Dumped server information to /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-2/data/info.pb
I20260810 01:57:45.620371 22804 external_mini_cluster.cc:1434] Started /tmp/dist-test-task_qhVVJ/build/asan/bin/kudu as pid 25602
I20260810 01:57:45.627403 25714 sys_catalog.cc:263] Verifying existing consensus state
I20260810 01:57:45.630787 25714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721: Bootstrap starting.
I20260810 01:57:45.645174 25714 log.cc:826] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721: Log is configured to *not* fsync() on all Append() calls
I20260810 01:57:45.675771 25714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721: Bootstrap replayed 1/1 log segments. Stats: ops{read=40 overwritten=0 applied=40 ignored=0} inserts{seen=28 ignored=0} mutations{seen=27 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 01:57:45.676407 25714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721: Bootstrap complete.
I20260810 01:57:45.682156 25714 raft_consensus.cc:359] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 5 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:45.682876 25714 raft_consensus.cc:740] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 5 FOLLOWER]: Becoming Follower/Learner. State: Replica: 757bb3f336084510845ed58df264c721, State: Initialized, Role: FOLLOWER
I20260810 01:57:45.683566 25714 consensus_queue.cc:260] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 40, Last appended: 5.40, Last appended by leader: 40, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:45.684341 25721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 5 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:45.684613 25721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:57:45.685412 25714 sys_catalog.cc:565] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:57:45.698781 25743 catalog_manager.cc:1295] Loaded cluster ID: dfa48c56b63b4d0cbf3d6124921e8a17
I20260810 01:57:45.698879 25743 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721: loading cluster ID for follower catalog manager: success
I20260810 01:57:45.694679 25602 master_runner.cc:187] Error getting master registration for 127.22.69.62:35311: Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111),
I20260810 01:57:45.700879 25602 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 01:57:45.703091 25743 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721: acquiring CA information for follower catalog manager: success
I20260810 01:57:45.705806 25743 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 01:57:45.762038 24063 heartbeater.cc:344] Connected to a master server at 127.22.69.60:36917
I20260810 01:57:45.765100 25664 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" instance_seqno: 1786327048775850) as {username='slave'} at 127.22.69.2:56409; Asking this server to re-register.
I20260810 01:57:45.765889 24063 heartbeater.cc:461] Registering TS with master...
I20260810 01:57:45.767741 25664 ts_manager.cc:194] Registered new tserver with Master: 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
I20260810 01:57:45.824182 23997 raft_consensus.cc:1217] T 4514cc930ba44e0f92a438f8ae0ba6fc P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 0.0->[1.1-1.1] Dedup: 1.1->[]
I20260810 01:57:45.830765 23841 heartbeater.cc:344] Connected to a master server at 127.22.69.60:36917
I20260810 01:57:45.832439 25664 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" instance_seqno: 1786327048176732) as {username='slave'} at 127.22.69.1:35739; Asking this server to re-register.
I20260810 01:57:45.833093 23841 heartbeater.cc:461] Registering TS with master...
I20260810 01:57:45.834457 25664 ts_manager.cc:194] Registered new tserver with Master: 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491)
I20260810 01:57:46.067114 24298 heartbeater.cc:344] Connected to a master server at 127.22.69.60:36917
I20260810 01:57:46.067344 24298 heartbeater.cc:499] Master 127.22.69.60:36917 was elected leader, sending a full tablet report...
I20260810 01:57:46.069314 25664 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "498d352dbea9405199a121e84a4ab926" instance_seqno: 1786327049319804) as {username='slave'} at 127.22.69.3:36321; Asking this server to re-register.
I20260810 01:57:46.070382 24298 heartbeater.cc:461] Registering TS with master...
I20260810 01:57:46.071660 25664 ts_manager.cc:194] Registered new tserver with Master: 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
I20260810 01:57:46.305140 25767 raft_consensus.cc:493] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:57:46.305413 25767 raft_consensus.cc:515] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:46.307304 25767 leader_election.cc:290] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), b1422349180c446db6c1768b3025253c (127.22.69.61:41945)
W20260810 01:57:46.308229 25644 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111)
W20260810 01:57:46.310636 25644 leader_election.cc:336] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311): Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111)
I20260810 01:57:46.314802 25189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "757bb3f336084510845ed58df264c721" candidate_term: 6 candidate_status { last_received { term: 5 index: 40 } } ignore_live_leader: false dest_uuid: "b1422349180c446db6c1768b3025253c" is_pre_election: true
I20260810 01:57:46.315045 25189 raft_consensus.cc:2468] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 757bb3f336084510845ed58df264c721 in term 5.
I20260810 01:57:46.315744 25639 leader_election.cc:304] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c; no voters: 919313c0dee540a690f46dbc8208d20b
I20260810 01:57:46.316149 25767 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260810 01:57:46.316296 25767 raft_consensus.cc:493] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 01:57:46.316401 25767 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 5 FOLLOWER]: Advancing to term 6
I20260810 01:57:46.320025 25767 raft_consensus.cc:515] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:46.320562 25767 leader_election.cc:290] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [CANDIDATE]: Term 6 election: Requested vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), b1422349180c446db6c1768b3025253c (127.22.69.61:41945)
I20260810 01:57:46.321108 25189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "757bb3f336084510845ed58df264c721" candidate_term: 6 candidate_status { last_received { term: 5 index: 40 } } ignore_live_leader: false dest_uuid: "b1422349180c446db6c1768b3025253c"
I20260810 01:57:46.321337 25189 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 5 FOLLOWER]: Advancing to term 6
W20260810 01:57:46.321947 25644 leader_election.cc:336] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311): Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111)
I20260810 01:57:46.323668 25189 raft_consensus.cc:2468] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 757bb3f336084510845ed58df264c721 in term 6.
I20260810 01:57:46.324101 25639 leader_election.cc:304] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c; no voters: 919313c0dee540a690f46dbc8208d20b
I20260810 01:57:46.324432 25767 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 6 FOLLOWER]: Leader election won for term 6
I20260810 01:57:46.324855 25767 raft_consensus.cc:697] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [term 6 LEADER]: Becoming Leader. State: Replica: 757bb3f336084510845ed58df264c721, State: Running, Role: LEADER
I20260810 01:57:46.325322 25767 consensus_queue.cc:237] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 40, Committed index: 40, Last appended: 5.40, Last appended by leader: 40, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:46.328380 25772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 757bb3f336084510845ed58df264c721. Latest consensus state: current_term: 6 leader_uuid: "757bb3f336084510845ed58df264c721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:46.328606 25772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [sys.catalog]: This master's current role is: LEADER
I20260810 01:57:46.329272 25775 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260810 01:57:46.331732 25775 catalog_manager.cc:679] Loaded metadata for table default.table_72b6f93a124042e4a090dc194134c75a [id=017756566b72415aa33842f8df2b3e91]
I20260810 01:57:46.332110 25775 catalog_manager.cc:679] Loaded metadata for table default.table_73d521a2c5504815ad6d7cc3dd0bf818 [id=2f1832807601444a88d61b8396a67db9]
I20260810 01:57:46.332413 25775 catalog_manager.cc:679] Loaded metadata for table default.table_c8ad906be0f74210912a57adead21255 [id=4da3c920ea6a48588c9f0c91da00fb69]
I20260810 01:57:46.332842 25775 catalog_manager.cc:679] Loaded metadata for table default.table_983bf2e604df4fdbac75c5aca292f736 [id=552771dde220474a9be9abeefd3a0472]
I20260810 01:57:46.333189 25775 catalog_manager.cc:679] Loaded metadata for table default.table_22726fd1a11f409f9f5287ab16d8e551 [id=5edf9cb336414ae3bd41e9f5ca41fcca]
I20260810 01:57:46.333564 25775 catalog_manager.cc:679] Loaded metadata for table default.table_0ba2046f969346f68f1bde7cf6c9f273 [id=630b03060b7646118927a18573c0920c]
I20260810 01:57:46.333899 25775 catalog_manager.cc:679] Loaded metadata for table default.table_510ae1b5fcc74db788ffb64925cad250 [id=7d240d10089c47eaa014d59c4ac7f0b4]
I20260810 01:57:46.334239 25775 catalog_manager.cc:679] Loaded metadata for table default.table_379ea52611ef49d591a022addb7afc0c [id=87ba140e9080402aa494198d6f212f05]
I20260810 01:57:46.334578 25775 catalog_manager.cc:679] Loaded metadata for table default.table_57697192c9bf44969c2b200e94865a3e [id=ef2e836fffc64aa6a8ddc0425f39e317]
I20260810 01:57:46.337348 25775 tablet_loader.cc:96] loaded metadata for tablet 1c3db8429f9f4cf5b29d3d1a6cc0651c (table default.table_379ea52611ef49d591a022addb7afc0c [id=87ba140e9080402aa494198d6f212f05])
I20260810 01:57:46.337685 25775 tablet_loader.cc:96] loaded metadata for tablet 2230d919953e49208659450a024c6a62 (table default.table_0ba2046f969346f68f1bde7cf6c9f273 [id=630b03060b7646118927a18573c0920c])
I20260810 01:57:46.337996 25775 tablet_loader.cc:96] loaded metadata for tablet 4514cc930ba44e0f92a438f8ae0ba6fc (table default.table_510ae1b5fcc74db788ffb64925cad250 [id=7d240d10089c47eaa014d59c4ac7f0b4])
I20260810 01:57:46.338292 25775 tablet_loader.cc:96] loaded metadata for tablet 4579a90be0b6409781fd9e470324192c (table default.table_22726fd1a11f409f9f5287ab16d8e551 [id=5edf9cb336414ae3bd41e9f5ca41fcca])
I20260810 01:57:46.338613 25775 tablet_loader.cc:96] loaded metadata for tablet 5c49a207b37e40579d34002c3fb57afb (table default.table_c8ad906be0f74210912a57adead21255 [id=4da3c920ea6a48588c9f0c91da00fb69])
I20260810 01:57:46.338920 25775 tablet_loader.cc:96] loaded metadata for tablet 6ca335f1e5964f8490a49ba4c4fb4c72 (table default.table_72b6f93a124042e4a090dc194134c75a [id=017756566b72415aa33842f8df2b3e91])
I20260810 01:57:46.339424 25775 tablet_loader.cc:96] loaded metadata for tablet a48e9f780b4e49abafaadbd78ab8711a (table default.table_57697192c9bf44969c2b200e94865a3e [id=ef2e836fffc64aa6a8ddc0425f39e317])
I20260810 01:57:46.340001 25775 tablet_loader.cc:96] loaded metadata for tablet cad2404759604bfea7c273e1aa40539a (table default.table_73d521a2c5504815ad6d7cc3dd0bf818 [id=2f1832807601444a88d61b8396a67db9])
I20260810 01:57:46.340317 25775 tablet_loader.cc:96] loaded metadata for tablet e6ef3afe40a847dbb08792578d52401d (table default.table_983bf2e604df4fdbac75c5aca292f736 [id=552771dde220474a9be9abeefd3a0472])
I20260810 01:57:46.340570 25775 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260810 01:57:46.341465 25775 catalog_manager.cc:1295] Loaded cluster ID: dfa48c56b63b4d0cbf3d6124921e8a17
I20260810 01:57:46.341583 25775 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260810 01:57:46.343099 25775 catalog_manager.cc:1540] Loading token signing keys...
I20260810 01:57:46.344357 25775 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721: Loaded TSK: 0
I20260810 01:57:46.345191 25775 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260810 01:57:46.346052 25775 catalog_manager.cc:1563] Loading latest processed Hive Metastore notification log event ID...
I20260810 01:57:46.346860 25775 catalog_manager.cc:5941] Last processed Hive Metastore notification event ID: -1
I20260810 01:57:46.429458 25664 master_service.cc:960] ReplaceTablet: received request to replace tablet 1844a63489d74f1c919e96501e4bd568 from {username='slave'} at 127.0.0.1:60128
I20260810 01:57:46.429435 25663 master_service.cc:960] ReplaceTablet: received request to replace tablet 16db112abfb34d4a9f2ecbee98c39d8b from {username='slave'} at 127.0.0.1:60128
I20260810 01:57:46.429617 25661 master_service.cc:960] ReplaceTablet: received request to replace tablet e6ef3afe40a847dbb08792578d52401d from {username='slave'} at 127.0.0.1:60128
I20260810 01:57:46.430289 25660 master_service.cc:960] ReplaceTablet: received request to replace tablet 89d9556cc526444299c786de606bb5b1 from {username='slave'} at 127.0.0.1:60128
I20260810 01:57:46.430845 25659 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "default.Table_5a4f0e1bcc16485ab9a7148d13d21a68"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:46.431905 25656 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "default.Table_22726fd1a11f409f9f5287ab16d8e551" } modify_external_catalogs: true
I20260810 01:57:46.432137 25663 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "default.Table_e3c77b251fdf471d8916f06da94be990"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:46.432199 25656 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "default.Table_22726fd1a11f409f9f5287ab16d8e551" } modify_external_catalogs: true
I20260810 01:57:46.432767 25660 master_service.cc:960] ReplaceTablet: received request to replace tablet 1844a63489d74f1c919e96501e4bd568 from {username='slave'} at 127.0.0.1:60128
I20260810 01:57:46.433558 25664 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "default.Table_14c5bada100e48adb5f378e73ee7d00b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:46.433499 25660 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "default.Table_b03802d8f2f14a24b00ab6efbe9392dc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:46.434409 25654 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "default.Table_57697192c9bf44969c2b200e94865a3e" } modify_external_catalogs: true
I20260810 01:57:46.434801 25655 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "default.Table_0ba2046f969346f68f1bde7cf6c9f273" } modify_external_catalogs: true
I20260810 01:57:46.435156 25652 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "default.Table_510ae1b5fcc74db788ffb64925cad250" } modify_external_catalogs: true
I20260810 01:57:46.441062 25654 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "default.Table_57697192c9bf44969c2b200e94865a3e" } modify_external_catalogs: true
I20260810 01:57:46.441145 25652 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "default.Table_510ae1b5fcc74db788ffb64925cad250" } modify_external_catalogs: true
I20260810 01:57:46.441062 25655 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "default.Table_0ba2046f969346f68f1bde7cf6c9f273" } modify_external_catalogs: true
I20260810 01:57:46.435253 25653 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "default.Table_7e5efb367a02411aac568a75dd39b274"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260810 01:57:46.441859 25742 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 10 received events: NotificationEvent(eventId=1, eventTime=1786327049, eventType=CREATE_TABLE, dbName=default, tableName=table_4fb63c687c11493ca7c0b76735f137e9, message={"server":"","servicePrincipal":"","db":"default","table":"table_4fb63c687c11493ca7c0b76735f137e9","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_4fb63c687c11493ca7c0b76735f137e9\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1786327049},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/warehouse/table_4fb63c687c11493ca7c0b76735f137e9\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_4fb63c687c11493ca7c0b76735f137e9\",\"kudu.table_id\":\"cb924727e6014a2a9dd271f962df3807\",\"transient_lastDdlTime\":\"1786327049\",\"comment\":\"\",\"kudu.master_addresses\":\"127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"dfa48c56b63b4d0cbf3d6124921e8a17\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1786327049,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1786327049, eventType=CREATE_TABLE, dbName=default, tableName=table_c8ad906be0f74210912a57adead21255, message={"server":"","servicePrincipal":"","db":"default","table":"table_c8ad906be0f74210912a57adead21255","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_c8ad906be0f74210912a57adead21255\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1786327049},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/warehouse/table_c8ad906be0f74210912a57adead21255\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_c8ad906be0f74210912a57adead21255\",\"kudu.table_id\":\"4da3c920ea6a48588c9f0c91da00fb69\",\"transient_lastDdlTime\":\"1786327049\",\"comment\":\"\",\"kudu.master_addresses\":\"127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"dfa48c56b63b4d0cbf3d6124921e8a17\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1786327049,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
I20260810 01:57:46.445039 25189 raft_consensus.cc:1275] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 6 FOLLOWER]: Refusing update from remote peer 757bb3f336084510845ed58df264c721: Log matching property violated. Preceding OpId in replica: term: 5 index: 40. Preceding OpId from leader: term: 6 index: 42. (index mismatch)
@ 0x7f2d09d13d5d google::LogMessage::Fail() at ??:0
W20260810 01:57:46.446215 25644 consensus_peers.cc:597] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 -> Peer 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311): Couldn't send request to peer 919313c0dee540a690f46dbc8208d20b. Status: Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 01:57:46.446807 25772 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 757bb3f336084510845ed58df264c721 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 41, Last known committed idx: 40, Time since last communication: 0.000s
@ 0x7f2d09d17b23 google::LogMessage::SendToLog() at ??:0
@ 0x7f2d09d1375c google::LogMessage::Flush() at ??:0
@ 0x7f2d09d139a1 google::LogMessage::~LogMessage() at ??:0
@ 0x7f2d209dcf64 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7f2d209db811 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f2d209e348c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7f2d209e3309 std::_Function_handler<>::_M_invoke() at ??:0
@ 0xaaf86d std::function<>::operator()() at ??:?
@ 0x7f2d0ca3bdd9 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f2d1010b6db start_thread at ??:0
@ 0x7f2d08bc261f clone at ??:0
I20260810 01:57:46.485913 25779 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 757bb3f336084510845ed58df264c721. Latest consensus state: current_term: 6 leader_uuid: "757bb3f336084510845ed58df264c721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:46.486245 25779 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: This master's current role is: FOLLOWER
W20260810 01:57:46.661422 24111 connection.cc:573] client connection to 127.22.69.60:36917 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 01:57:46.661443 25144 connection.cc:573] server connection from 127.0.0.1:59792 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 01:57:46.661919 24317 connection.cc:573] client connection to 127.22.69.60:36917 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 01:57:46.662052 23890 connection.cc:573] client connection to 127.22.69.60:36917 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 01:57:46.662935 24317 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.22.69.60:36917)
W20260810 01:57:46.663489 24317 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.22.69.60:36917)
W20260810 01:57:46.663686 24317 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.22.69.60:36917)
W20260810 01:57:46.664103 24317 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.22.69.60:36917)
I20260810 01:57:46.680191 25601 raft_consensus.cc:493] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:57:46.680413 25601 raft_consensus.cc:515] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:46.681278 25601 leader_election.cc:290] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:57:46.682188 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
W20260810 01:57:46.682408 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
I20260810 01:57:46.682551 24109 leader_election.cc:304] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:57:46.682834 25601 raft_consensus.cc:2749] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:57:46.961555 25779 raft_consensus.cc:493] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 757bb3f336084510845ed58df264c721)
I20260810 01:57:46.961763 25779 raft_consensus.cc:515] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:46.962496 25779 leader_election.cc:290] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:57:46.964701 25145 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311): Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111)
W20260810 01:57:46.964924 25142 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:57:46.965050 25142 leader_election.cc:304] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1422349180c446db6c1768b3025253c; no voters: 757bb3f336084510845ed58df264c721, 919313c0dee540a690f46dbc8208d20b
I20260810 01:57:46.965307 25779 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260810 01:57:47.084376 25601 raft_consensus.cc:493] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:57:47.084550 25601 raft_consensus.cc:515] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:47.085206 25601 leader_election.cc:290] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:57:47.086022 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
W20260810 01:57:47.086273 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
I20260810 01:57:47.086378 24109 leader_election.cc:304] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:57:47.086648 25601 raft_consensus.cc:2749] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:57:47.486904 25816 raft_consensus.cc:493] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 757bb3f336084510845ed58df264c721)
I20260810 01:57:47.487162 25816 raft_consensus.cc:515] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:47.487859 25816 leader_election.cc:290] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:57:47.490191 25145 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311): Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111)
W20260810 01:57:47.490998 25142 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:57:47.491164 25142 leader_election.cc:304] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1422349180c446db6c1768b3025253c; no voters: 757bb3f336084510845ed58df264c721, 919313c0dee540a690f46dbc8208d20b
I20260810 01:57:47.491402 25816 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260810 01:57:47.775666 25601 raft_consensus.cc:493] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:57:47.775835 25601 raft_consensus.cc:515] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:47.776366 25601 leader_election.cc:290] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:57:47.777243 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
W20260810 01:57:47.777573 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
I20260810 01:57:47.777732 24109 leader_election.cc:304] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:57:47.778043 25601 raft_consensus.cc:2749] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:57:48.362300 25834 raft_consensus.cc:493] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 757bb3f336084510845ed58df264c721)
I20260810 01:57:48.362537 25834 raft_consensus.cc:515] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:48.363276 25834 leader_election.cc:290] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:57:48.365522 25142 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
W20260810 01:57:48.365765 25145 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311): Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111)
I20260810 01:57:48.365899 25145 leader_election.cc:304] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1422349180c446db6c1768b3025253c; no voters: 757bb3f336084510845ed58df264c721, 919313c0dee540a690f46dbc8208d20b
I20260810 01:57:48.366163 25834 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260810 01:57:48.382030 22804 external_mini_cluster.cc:1372] Running /tmp/dist-test-task_qhVVJ/build/asan/bin/kudu
/tmp/dist-test-task_qhVVJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.69.62:35311
--webserver_interface=127.22.69.62
--webserver_port=40455
--builtin_ntp_servers=127.22.69.20:43209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917
--hive_metastore_uris=thrift://127.0.0.1:37513
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260810 01:57:48.510097 25117 master_runner.cc:187] Error getting master registration for 127.22.69.62:35311: Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111),
I20260810 01:57:48.510365 25117 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 01:57:48.584831 25830 raft_consensus.cc:493] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 01:57:48.585101 25830 raft_consensus.cc:515] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:48.586462 25830 leader_election.cc:290] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 01:57:48.587237 23889 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
W20260810 01:57:48.587543 23892 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
I20260810 01:57:48.587692 23892 leader_election.cc:304] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:57:48.587982 25830 raft_consensus.cc:2749] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 01:57:48.726179 25840 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:57:48.726512 25840 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:57:48.726661 25840 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260810 01:57:48.735013 25840 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:57:48.735149 25840 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:57:48.735258 25840 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:57:48.735342 25840 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:57:48.745476 25840 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.69.20:43209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:37513
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.69.62:35311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.69.62
--webserver_port=40455
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 10 Aug 2026 01:44:04 UTC on e7f111948823
build id 13266
ASAN enabled
I20260810 01:57:48.747166 25840 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:57:48.748984 25840 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260810 01:57:48.757043 25848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:57:48.757084 25849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:57:48.757745 25854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:57:48.758561 25840 server_base.cc:1061] running on GCE node
I20260810 01:57:48.759321 25840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:57:48.760711 25840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:57:48.761950 25840 hybrid_clock.cc:648] HybridClock initialized: now 1786327068761916 us; error 42 us; skew 500 ppm
I20260810 01:57:48.762367 25840 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 01:57:48.764757 25840 webserver.cc:533] Webserver started at http://127.22.69.62:40455/ using document root <none> and password file <none>
I20260810 01:57:48.765416 25840 fs_manager.cc:362] Metadata directory not provided
I20260810 01:57:48.765514 25840 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:57:48.770927 25840 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.001s sys 0.001s
I20260810 01:57:48.773726 25862 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:57:48.775247 25840 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 01:57:48.775415 25840 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/data,/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/wal
uuid: "919313c0dee540a690f46dbc8208d20b"
format_stamp: "Formatted at 2026-08-10 01:57:26 on dist-test-slave-n326"
I20260810 01:57:48.776149 25840 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:57:48.791028 25840 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260810 01:57:48.791884 25840 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:57:48.792196 25840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:57:48.812389 25840 rpc_server.cc:307] RPC server started. Bound to: 127.22.69.62:35311
I20260810 01:57:48.812408 25936 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.22.69.62:35311 every 8 connection(s)
I20260810 01:57:48.814042 25840 server_base.cc:1193] Dumped server information to /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/data/info.pb
I20260810 01:57:48.817145 22804 external_mini_cluster.cc:1434] Started /tmp/dist-test-task_qhVVJ/build/asan/bin/kudu as pid 25840
I20260810 01:57:48.822480 25940 sys_catalog.cc:263] Verifying existing consensus state
I20260810 01:57:48.827028 25940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b: Bootstrap starting.
I20260810 01:57:48.842900 23842 heartbeater.cc:344] Connected to a master server at 127.22.69.62:35311
I20260810 01:57:48.845471 25940 log.cc:826] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b: Log is configured to *not* fsync() on all Append() calls
I20260810 01:57:48.876366 25940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b: Bootstrap replayed 1/1 log segments. Stats: ops{read=34 overwritten=0 applied=32 ignored=0} inserts{seen=24 ignored=0} mutations{seen=19 ignored=0} orphaned_commits=0. Pending: 2 replicates
I20260810 01:57:48.876942 25940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b: Bootstrap complete.
I20260810 01:57:48.883337 25952 raft_consensus.cc:493] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 757bb3f336084510845ed58df264c721)
I20260810 01:57:48.883651 25952 raft_consensus.cc:515] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:48.883364 25940 raft_consensus.cc:359] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 3 FOLLOWER]: Replica starting. Triggering 2 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:48.888032 25940 raft_consensus.cc:740] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 919313c0dee540a690f46dbc8208d20b, State: Initialized, Role: FOLLOWER
I20260810 01:57:48.889088 25952 leader_election.cc:290] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
I20260810 01:57:48.889097 25940 consensus_queue.cc:260] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 32, Last appended: 3.34, Last appended by leader: 34, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:48.890601 25964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:48.891153 25964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:57:48.895828 25940 sys_catalog.cc:565] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:57:48.896198 25899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b1422349180c446db6c1768b3025253c" candidate_term: 7 candidate_status { last_received { term: 6 index: 42 } } ignore_live_leader: false dest_uuid: "919313c0dee540a690f46dbc8208d20b" is_pre_election: true
I20260810 01:57:48.896405 24066 heartbeater.cc:344] Connected to a master server at 127.22.69.62:35311
I20260810 01:57:48.896713 25899 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1422349180c446db6c1768b3025253c in term 3.
I20260810 01:57:48.897395 25145 leader_election.cc:304] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b, b1422349180c446db6c1768b3025253c; no voters:
I20260810 01:57:48.897683 25952 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260810 01:57:48.897828 25952 raft_consensus.cc:493] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 757bb3f336084510845ed58df264c721)
I20260810 01:57:48.897943 25952 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 6 FOLLOWER]: Advancing to term 7
W20260810 01:57:48.898228 25142 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:57:48.900704 25952 raft_consensus.cc:515] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:48.901387 25952 leader_election.cc:290] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 7 election: Requested vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
I20260810 01:57:48.901765 25899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b1422349180c446db6c1768b3025253c" candidate_term: 7 candidate_status { last_received { term: 6 index: 42 } } ignore_live_leader: false dest_uuid: "919313c0dee540a690f46dbc8208d20b"
I20260810 01:57:48.901969 25899 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 3 FOLLOWER]: Advancing to term 7
W20260810 01:57:48.903244 25142 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:57:48.906701 25899 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1422349180c446db6c1768b3025253c in term 7.
I20260810 01:57:48.907226 25145 leader_election.cc:304] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b, b1422349180c446db6c1768b3025253c; no voters: 757bb3f336084510845ed58df264c721
I20260810 01:57:48.907740 25952 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 7 FOLLOWER]: Leader election won for term 7
I20260810 01:57:48.909442 25952 raft_consensus.cc:697] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 7 LEADER]: Becoming Leader. State: Replica: b1422349180c446db6c1768b3025253c, State: Running, Role: LEADER
I20260810 01:57:48.909952 25952 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 40, Committed index: 40, Last appended: 6.42, Last appended by leader: 42, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:48.923735 25986 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1422349180c446db6c1768b3025253c. Latest consensus state: current_term: 7 leader_uuid: "b1422349180c446db6c1768b3025253c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:48.924085 25986 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: This master's current role is: LEADER
I20260810 01:57:48.928121 25990 catalog_manager.cc:1295] Loaded cluster ID: dfa48c56b63b4d0cbf3d6124921e8a17
I20260810 01:57:48.928256 25990 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b: loading cluster ID for follower catalog manager: success
I20260810 01:57:48.932456 25990 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b: acquiring CA information for follower catalog manager: success
I20260810 01:57:48.935259 25990 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 01:57:48.962565 25840 master_runner.cc:187] Error getting master registration for 127.22.69.60:36917: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111),
I20260810 01:57:48.966504 25840 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 01:57:49.075440 24300 heartbeater.cc:344] Connected to a master server at 127.22.69.62:35311
I20260810 01:57:49.078384 25889 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "498d352dbea9405199a121e84a4ab926" instance_seqno: 1786327049319804) as {username='slave'} at 127.22.69.3:55749; Asking this server to re-register.
I20260810 01:57:49.079185 24300 heartbeater.cc:461] Registering TS with master...
I20260810 01:57:49.080880 25889 ts_manager.cc:194] Registered new tserver with Master: 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
I20260810 01:57:49.350416 25889 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" instance_seqno: 1786327048176732) as {username='slave'} at 127.22.69.1:32949; Asking this server to re-register.
I20260810 01:57:49.351166 23842 heartbeater.cc:461] Registering TS with master...
I20260810 01:57:49.352483 25889 ts_manager.cc:194] Registered new tserver with Master: 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491)
I20260810 01:57:49.402863 25889 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" instance_seqno: 1786327048775850) as {username='slave'} at 127.22.69.2:34181; Asking this server to re-register.
I20260810 01:57:49.404002 24066 heartbeater.cc:461] Registering TS with master...
I20260810 01:57:49.405325 25889 ts_manager.cc:194] Registered new tserver with Master: 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:57:49.443784 25142 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111) [suppressed 11 similar messages]
W20260810 01:57:49.444972 25142 consensus_peers.cc:597] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c -> Peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Couldn't send request to peer 757bb3f336084510845ed58df264c721. Status: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 01:57:49.528194 25899 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 7 FOLLOWER]: Refusing update from remote peer b1422349180c446db6c1768b3025253c: Log matching property violated. Preceding OpId in replica: term: 3 index: 34. Preceding OpId from leader: term: 7 index: 43. (index mismatch)
I20260810 01:57:49.529110 26009 consensus_queue.cc:1048] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [LEADER]: Connected to new peer: Peer: permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 43, Last known committed idx: 32, Time since last communication: 0.000s
I20260810 01:57:49.537305 26020 mvcc.cc:204] Tried to move back new op lower bound from 7316795649478799360 to 7316795620370968576. Current Snapshot: MvccSnapshot[applied={T|T < 7316795649478799360}]
I20260810 01:57:49.537230 26026 sys_catalog.cc:455] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "b1422349180c446db6c1768b3025253c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:49.537429 26023 sys_catalog.cc:455] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "b1422349180c446db6c1768b3025253c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:49.537614 26026 sys_catalog.cc:458] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:57:49.537690 26023 sys_catalog.cc:458] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:57:49.537240 26027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "b1422349180c446db6c1768b3025253c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:49.538256 26027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:57:49.537976 26017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1422349180c446db6c1768b3025253c. Latest consensus state: current_term: 7 leader_uuid: "b1422349180c446db6c1768b3025253c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:49.538491 26017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:57:49.540833 26027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "b1422349180c446db6c1768b3025253c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:49.540948 26017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "b1422349180c446db6c1768b3025253c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:49.541030 26027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:57:49.540944 26009 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "b1422349180c446db6c1768b3025253c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:49.541115 26017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:57:49.541209 26009 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: This master's current role is: LEADER
I20260810 01:57:49.541601 26035 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 7 leader_uuid: "b1422349180c446db6c1768b3025253c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:49.541776 26035 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: This master's current role is: LEADER
I20260810 01:57:49.573730 26008 raft_consensus.cc:493] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 01:57:49.573885 26008 raft_consensus.cc:515] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:49.574543 26008 leader_election.cc:290] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 01:57:49.575515 23889 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
W20260810 01:57:49.575686 23892 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
I20260810 01:57:49.575829 23892 leader_election.cc:304] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:57:49.576103 26008 raft_consensus.cc:2749] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 01:57:49.586992 26000 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260810 01:57:49.589363 26000 catalog_manager.cc:679] Loaded metadata for table default.table_72b6f93a124042e4a090dc194134c75a [id=017756566b72415aa33842f8df2b3e91]
I20260810 01:57:49.589933 26000 catalog_manager.cc:679] Loaded metadata for table default.table_73d521a2c5504815ad6d7cc3dd0bf818 [id=2f1832807601444a88d61b8396a67db9]
I20260810 01:57:49.590327 26000 catalog_manager.cc:679] Loaded metadata for table default.table_c8ad906be0f74210912a57adead21255 [id=4da3c920ea6a48588c9f0c91da00fb69]
I20260810 01:57:49.590695 26000 catalog_manager.cc:679] Loaded metadata for table default.table_983bf2e604df4fdbac75c5aca292f736 [id=552771dde220474a9be9abeefd3a0472]
I20260810 01:57:49.591095 26000 catalog_manager.cc:679] Loaded metadata for table default.table_22726fd1a11f409f9f5287ab16d8e551 [id=5edf9cb336414ae3bd41e9f5ca41fcca]
I20260810 01:57:49.591495 26000 catalog_manager.cc:679] Loaded metadata for table default.table_0ba2046f969346f68f1bde7cf6c9f273 [id=630b03060b7646118927a18573c0920c]
I20260810 01:57:49.591866 26000 catalog_manager.cc:679] Loaded metadata for table default.table_510ae1b5fcc74db788ffb64925cad250 [id=7d240d10089c47eaa014d59c4ac7f0b4]
I20260810 01:57:49.592231 26000 catalog_manager.cc:679] Loaded metadata for table default.table_379ea52611ef49d591a022addb7afc0c [id=87ba140e9080402aa494198d6f212f05]
I20260810 01:57:49.592578 26000 catalog_manager.cc:679] Loaded metadata for table default.table_57697192c9bf44969c2b200e94865a3e [id=ef2e836fffc64aa6a8ddc0425f39e317]
I20260810 01:57:49.594816 26000 tablet_loader.cc:96] loaded metadata for tablet 1c3db8429f9f4cf5b29d3d1a6cc0651c (table default.table_379ea52611ef49d591a022addb7afc0c [id=87ba140e9080402aa494198d6f212f05])
I20260810 01:57:49.595225 26000 tablet_loader.cc:96] loaded metadata for tablet 2230d919953e49208659450a024c6a62 (table default.table_0ba2046f969346f68f1bde7cf6c9f273 [id=630b03060b7646118927a18573c0920c])
I20260810 01:57:49.595577 26000 tablet_loader.cc:96] loaded metadata for tablet 4514cc930ba44e0f92a438f8ae0ba6fc (table default.table_510ae1b5fcc74db788ffb64925cad250 [id=7d240d10089c47eaa014d59c4ac7f0b4])
I20260810 01:57:49.595890 26000 tablet_loader.cc:96] loaded metadata for tablet 4579a90be0b6409781fd9e470324192c (table default.table_22726fd1a11f409f9f5287ab16d8e551 [id=5edf9cb336414ae3bd41e9f5ca41fcca])
I20260810 01:57:49.596177 26000 tablet_loader.cc:96] loaded metadata for tablet 5974ae70170544f797ff749280b3be0c (table default.table_983bf2e604df4fdbac75c5aca292f736 [id=552771dde220474a9be9abeefd3a0472])
I20260810 01:57:49.596472 26000 tablet_loader.cc:96] loaded metadata for tablet 5c49a207b37e40579d34002c3fb57afb (table default.table_c8ad906be0f74210912a57adead21255 [id=4da3c920ea6a48588c9f0c91da00fb69])
I20260810 01:57:49.596815 26000 tablet_loader.cc:96] loaded metadata for tablet 6ca335f1e5964f8490a49ba4c4fb4c72 (table default.table_72b6f93a124042e4a090dc194134c75a [id=017756566b72415aa33842f8df2b3e91])
I20260810 01:57:49.597386 26000 tablet_loader.cc:96] loaded metadata for tablet a48e9f780b4e49abafaadbd78ab8711a (table default.table_57697192c9bf44969c2b200e94865a3e [id=ef2e836fffc64aa6a8ddc0425f39e317])
I20260810 01:57:49.598016 26000 tablet_loader.cc:96] loaded metadata for tablet cad2404759604bfea7c273e1aa40539a (table default.table_73d521a2c5504815ad6d7cc3dd0bf818 [id=2f1832807601444a88d61b8396a67db9])
I20260810 01:57:49.598472 26000 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260810 01:57:49.599408 26000 catalog_manager.cc:1295] Loaded cluster ID: dfa48c56b63b4d0cbf3d6124921e8a17
I20260810 01:57:49.599510 26000 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260810 01:57:49.600679 26000 catalog_manager.cc:1540] Loading token signing keys...
I20260810 01:57:49.601526 26000 catalog_manager.cc:6103] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c: Loaded TSK: 0
I20260810 01:57:49.602450 26000 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260810 01:57:49.603482 26000 catalog_manager.cc:1563] Loading latest processed Hive Metastore notification log event ID...
I20260810 01:57:49.604308 26000 catalog_manager.cc:5941] Last processed Hive Metastore notification event ID: -1
I20260810 01:57:49.660949 25169 master_service.cc:960] ReplaceTablet: received request to replace tablet 09ed68395e2d44a5b51a22abe2203406 from {username='slave'} at 127.0.0.1:59508
I20260810 01:57:49.661090 25170 master_service.cc:960] ReplaceTablet: received request to replace tablet 16db112abfb34d4a9f2ecbee98c39d8b from {username='slave'} at 127.0.0.1:59508
I20260810 01:57:49.662061 25167 master_service.cc:960] ReplaceTablet: received request to replace tablet 4579a90be0b6409781fd9e470324192c from {username='slave'} at 127.0.0.1:59508
I20260810 01:57:49.662472 25165 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:59508:
table { table_name: "default.Table_983bf2e604df4fdbac75c5aca292f736" } new_table_name: "default.Table_df285e759e764cdeb38de2aad0d1873d" modify_external_catalogs: true
I20260810 01:57:49.662953 25166 master_service.cc:960] ReplaceTablet: received request to replace tablet a48e9f780b4e49abafaadbd78ab8711a from {username='slave'} at 127.0.0.1:59508
I20260810 01:57:49.663470 25160 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59508:
name: "default.Table_73822ba0bac443d19cc7c66f6f547bda"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:49.664357 25157 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:59508:
table { table_name: "default.Table_c8ad906be0f74210912a57adead21255" } new_table_name: "default.Table_e2cb378ea960473c833b1d18fbe32f91" modify_external_catalogs: true
I20260810 01:57:49.665303 25152 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59508:
name: "default.Table_70288f170151490fb6ba63a3bf29f01e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:49.665668 25170 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:59508:
table { table_name: "default.Table_73d521a2c5504815ad6d7cc3dd0bf818" } new_table_name: "default.Table_f5851b3fa264482388e4e50f34d4e37c" modify_external_catalogs: true
I20260810 01:57:49.665007 25161 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:59508:
table { table_name: "default.Table_379ea52611ef49d591a022addb7afc0c" } new_table_name: "default.Table_09a7f5150ded428991a5d464657e85ae" modify_external_catalogs: true
I20260810 01:57:49.667179 25169 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:59508:
table { table_name: "default.Table_72b6f93a124042e4a090dc194134c75a" } new_table_name: "default.Table_b8900cd2c2574605ade34fc7ccc46f94" modify_external_catalogs: true
I20260810 01:57:49.667019 25164 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59508:
name: "default.Table_9175c2bd9b0542f2a6b95eae8a882b7a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:49.673555 25166 catalog_manager.cc:6006] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c: Sending DeleteTablet for 3 replicas of tablet a48e9f780b4e49abafaadbd78ab8711a
I20260810 01:57:49.677702 25167 catalog_manager.cc:6006] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c: Sending DeleteTablet for 3 replicas of tablet 4579a90be0b6409781fd9e470324192c
F20260810 01:57:49.679566 25256 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 10 received events: NotificationEvent(eventId=1, eventTime=1786327049, eventType=CREATE_TABLE, dbName=default, tableName=table_4fb63c687c11493ca7c0b76735f137e9, message={"server":"","servicePrincipal":"","db":"default","table":"table_4fb63c687c11493ca7c0b76735f137e9","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_4fb63c687c11493ca7c0b76735f137e9\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1786327049},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/warehouse/table_4fb63c687c11493ca7c0b76735f137e9\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_4fb63c687c11493ca7c0b76735f137e9\",\"kudu.table_id\":\"cb924727e6014a2a9dd271f962df3807\",\"transient_lastDdlTime\":\"1786327049\",\"comment\":\"\",\"kudu.master_addresses\":\"127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"dfa48c56b63b4d0cbf3d6124921e8a17\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1786327049,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1786327049, eventType=CREATE_TABLE, dbName=default, tableName=table_c8ad906be0f74210912a57adead21255, message={"server":"","servicePrincipal":"","db":"default","table":"table_c8ad906be0f74210912a57adead21255","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_c8ad906be0f74210912a57adead21255\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1786327049},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/warehouse/table_c8ad906be0f74210912a57adead21255\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_c8ad906be0f74210912a57adead21255\",\"kudu.table_id\":\"4da3c920ea6a48588c9f0c91da00fb69\",\"transient_lastDdlTime\":\"1786327049\",\"comment\":\"\",\"kudu.master_addresses\":\"127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"dfa48c56b63b4d0cbf3d6124921e8a17\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1786327049,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
@ 0x7f88b5b1bd5d google::LogMessage::Fail() at ??:0
I20260810 01:57:49.682961 25167 catalog_manager.cc:6807] ReplaceTablet: tablet 4579a90be0b6409781fd9e470324192c deleted and replaced by tablet f88158dec2a449c2a2011b8fc042f0ac
I20260810 01:57:49.683498 25167 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59508:
table { table_name: "default.Table_22726fd1a11f409f9f5287ab16d8e551" } modify_external_catalogs: true
I20260810 01:57:49.683758 25167 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59508:
table { table_name: "default.Table_22726fd1a11f409f9f5287ab16d8e551" } modify_external_catalogs: true
@ 0x7f88b5b1fb23 google::LogMessage::SendToLog() at ??:0
I20260810 01:57:49.685724 25166 catalog_manager.cc:6807] ReplaceTablet: tablet a48e9f780b4e49abafaadbd78ab8711a deleted and replaced by tablet 81e090d01d8f43cf8cd38e376b7c18ad
I20260810 01:57:49.685926 23964 tablet_service.cc:1558] Processing DeleteTablet for tablet a48e9f780b4e49abafaadbd78ab8711a with delete_type TABLET_DATA_DELETED (replaced by tablet 81e090d01d8f43cf8cd38e376b7c18ad) from {username='slave'} at 127.0.0.1:43262
I20260810 01:57:49.686069 25166 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59508:
name: "default.Table_5a4f0e1bcc16485ab9a7148d13d21a68"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
@ 0x7f88b5b1b75c google::LogMessage::Flush() at ??:0
I20260810 01:57:49.687187 23963 tablet_service.cc:1558] Processing DeleteTablet for tablet 4579a90be0b6409781fd9e470324192c with delete_type TABLET_DATA_DELETED (replaced by tablet f88158dec2a449c2a2011b8fc042f0ac) from {username='slave'} at 127.0.0.1:43262
@ 0x7f88b5b1b9a1 google::LogMessage::~LogMessage() at ??:0
I20260810 01:57:49.695843 26051 tablet_replica.cc:333] T a48e9f780b4e49abafaadbd78ab8711a P 48e33bc1e8e94d4887bb64214d0b6c36: stopping tablet replica
I20260810 01:57:49.696070 26051 raft_consensus.cc:2243] T a48e9f780b4e49abafaadbd78ab8711a P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 LEADER]: Raft consensus shutting down.
I20260810 01:57:49.696620 26051 raft_consensus.cc:2272] T a48e9f780b4e49abafaadbd78ab8711a P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:57:49.697371 26051 ts_tablet_manager.cc:1916] T a48e9f780b4e49abafaadbd78ab8711a P 48e33bc1e8e94d4887bb64214d0b6c36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:57:49.704259 26051 ts_tablet_manager.cc:1929] T a48e9f780b4e49abafaadbd78ab8711a P 48e33bc1e8e94d4887bb64214d0b6c36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 01:57:49.704397 26051 log.cc:1199] T a48e9f780b4e49abafaadbd78ab8711a P 48e33bc1e8e94d4887bb64214d0b6c36: Deleting WAL directory at /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-1/wal/wals/a48e9f780b4e49abafaadbd78ab8711a
I20260810 01:57:49.704982 26051 ts_tablet_manager.cc:1950] T a48e9f780b4e49abafaadbd78ab8711a P 48e33bc1e8e94d4887bb64214d0b6c36: Deleting consensus metadata
I20260810 01:57:49.706559 26051 tablet_replica.cc:333] T 4579a90be0b6409781fd9e470324192c P 48e33bc1e8e94d4887bb64214d0b6c36: stopping tablet replica
I20260810 01:57:49.706532 25141 catalog_manager.cc:5050] TS 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): tablet a48e9f780b4e49abafaadbd78ab8711a (table default.table_57697192c9bf44969c2b200e94865a3e [id=ef2e836fffc64aa6a8ddc0425f39e317]) successfully deleted
@ 0x7f88cc7e4f64 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260810 01:57:49.706799 26051 raft_consensus.cc:2243] T 4579a90be0b6409781fd9e470324192c P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 01:57:49.707013 26051 raft_consensus.cc:2272] T 4579a90be0b6409781fd9e470324192c P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:57:49.707616 26051 ts_tablet_manager.cc:1916] T 4579a90be0b6409781fd9e470324192c P 48e33bc1e8e94d4887bb64214d0b6c36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:57:49.710626 23746 tablet_service.cc:1558] Processing DeleteTablet for tablet a48e9f780b4e49abafaadbd78ab8711a with delete_type TABLET_DATA_DELETED (replaced by tablet 81e090d01d8f43cf8cd38e376b7c18ad) from {username='slave'} at 127.0.0.1:42574
I20260810 01:57:49.710721 23747 tablet_service.cc:1558] Processing DeleteTablet for tablet 4579a90be0b6409781fd9e470324192c with delete_type TABLET_DATA_DELETED (replaced by tablet f88158dec2a449c2a2011b8fc042f0ac) from {username='slave'} at 127.0.0.1:42574
I20260810 01:57:49.711210 24185 tablet_service.cc:1558] Processing DeleteTablet for tablet a48e9f780b4e49abafaadbd78ab8711a with delete_type TABLET_DATA_DELETED (replaced by tablet 81e090d01d8f43cf8cd38e376b7c18ad) from {username='slave'} at 127.0.0.1:46950
I20260810 01:57:49.714756 26055 tablet_replica.cc:333] T a48e9f780b4e49abafaadbd78ab8711a P 498d352dbea9405199a121e84a4ab926: stopping tablet replica
I20260810 01:57:49.714946 24185 tablet_service.cc:1558] Processing DeleteTablet for tablet 4579a90be0b6409781fd9e470324192c with delete_type TABLET_DATA_DELETED (replaced by tablet f88158dec2a449c2a2011b8fc042f0ac) from {username='slave'} at 127.0.0.1:46950
I20260810 01:57:49.714982 26055 raft_consensus.cc:2243] T a48e9f780b4e49abafaadbd78ab8711a P 498d352dbea9405199a121e84a4ab926 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 01:57:49.715245 26055 raft_consensus.cc:2272] T a48e9f780b4e49abafaadbd78ab8711a P 498d352dbea9405199a121e84a4ab926 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:57:49.715831 26055 ts_tablet_manager.cc:1916] T a48e9f780b4e49abafaadbd78ab8711a P 498d352dbea9405199a121e84a4ab926: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:57:49.717240 26051 ts_tablet_manager.cc:1929] T 4579a90be0b6409781fd9e470324192c P 48e33bc1e8e94d4887bb64214d0b6c36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 01:57:49.717402 26051 log.cc:1199] T 4579a90be0b6409781fd9e470324192c P 48e33bc1e8e94d4887bb64214d0b6c36: Deleting WAL directory at /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-1/wal/wals/4579a90be0b6409781fd9e470324192c
@ 0x7f88cc7e3811 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f88cc7eb48c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260810 01:57:49.726940 23744 tablet_service.cc:1511] Processing CreateTablet for tablet f88158dec2a449c2a2011b8fc042f0ac (DEFAULT_TABLE table=default.table_22726fd1a11f409f9f5287ab16d8e551 [id=5edf9cb336414ae3bd41e9f5ca41fcca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:49.727145 23963 tablet_service.cc:1511] Processing CreateTablet for tablet 5974ae70170544f797ff749280b3be0c (DEFAULT_TABLE table=default.table_983bf2e604df4fdbac75c5aca292f736 [id=552771dde220474a9be9abeefd3a0472]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:49.727388 23747 tablet_service.cc:1511] Processing CreateTablet for tablet 81e090d01d8f43cf8cd38e376b7c18ad (DEFAULT_TABLE table=default.table_57697192c9bf44969c2b200e94865a3e [id=ef2e836fffc64aa6a8ddc0425f39e317]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:49.727558 23963 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5974ae70170544f797ff749280b3be0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:49.727548 23744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f88158dec2a449c2a2011b8fc042f0ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:49.727931 23747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81e090d01d8f43cf8cd38e376b7c18ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:49.731436 23745 tablet_service.cc:1511] Processing CreateTablet for tablet 5974ae70170544f797ff749280b3be0c (DEFAULT_TABLE table=default.table_983bf2e604df4fdbac75c5aca292f736 [id=552771dde220474a9be9abeefd3a0472]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:49.731962 23745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5974ae70170544f797ff749280b3be0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:49.735589 23964 tablet_service.cc:1511] Processing CreateTablet for tablet 81e090d01d8f43cf8cd38e376b7c18ad (DEFAULT_TABLE table=default.table_57697192c9bf44969c2b200e94865a3e [id=ef2e836fffc64aa6a8ddc0425f39e317]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:49.736147 23964 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81e090d01d8f43cf8cd38e376b7c18ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:49.739321 26055 ts_tablet_manager.cc:1929] T a48e9f780b4e49abafaadbd78ab8711a P 498d352dbea9405199a121e84a4ab926: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 01:57:49.739480 26055 log.cc:1199] T a48e9f780b4e49abafaadbd78ab8711a P 498d352dbea9405199a121e84a4ab926: Deleting WAL directory at /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-2/wal/wals/a48e9f780b4e49abafaadbd78ab8711a
I20260810 01:57:49.739477 23962 tablet_service.cc:1511] Processing CreateTablet for tablet f88158dec2a449c2a2011b8fc042f0ac (DEFAULT_TABLE table=default.table_22726fd1a11f409f9f5287ab16d8e551 [id=5edf9cb336414ae3bd41e9f5ca41fcca]), partition=RANGE (key) PARTITION UNBOUNDED
@ 0x7f88cc7eb309 std::_Function_handler<>::_M_invoke() at ??:0
I20260810 01:57:49.739928 23962 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f88158dec2a449c2a2011b8fc042f0ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:49.739996 26055 ts_tablet_manager.cc:1950] T a48e9f780b4e49abafaadbd78ab8711a P 498d352dbea9405199a121e84a4ab926: Deleting consensus metadata
@ 0xaaf86d std::function<>::operator()() at ??:?
I20260810 01:57:49.744782 26055 tablet_replica.cc:333] T 4579a90be0b6409781fd9e470324192c P 498d352dbea9405199a121e84a4ab926: stopping tablet replica
I20260810 01:57:49.745090 26055 raft_consensus.cc:2243] T 4579a90be0b6409781fd9e470324192c P 498d352dbea9405199a121e84a4ab926 [term 1 LEADER]: Raft consensus shutting down.
I20260810 01:57:49.745522 26055 raft_consensus.cc:2272] T 4579a90be0b6409781fd9e470324192c P 498d352dbea9405199a121e84a4ab926 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:57:49.746261 26055 ts_tablet_manager.cc:1916] T 4579a90be0b6409781fd9e470324192c P 498d352dbea9405199a121e84a4ab926: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:57:49.747248 26058 tablet_bootstrap.cc:492] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab: Bootstrap starting.
I20260810 01:57:49.749650 26058 tablet_bootstrap.cc:654] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:49.752583 26059 tablet_bootstrap.cc:492] T 5974ae70170544f797ff749280b3be0c P 48e33bc1e8e94d4887bb64214d0b6c36: Bootstrap starting.
@ 0x7f88b8843dd9 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f88bbf136db start_thread at ??:0
I20260810 01:57:49.759928 26059 tablet_bootstrap.cc:654] T 5974ae70170544f797ff749280b3be0c P 48e33bc1e8e94d4887bb64214d0b6c36: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:49.762681 26058 tablet_bootstrap.cc:492] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab: No bootstrap required, opened a new log
I20260810 01:57:49.762828 26058 ts_tablet_manager.cc:1403] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.000s
I20260810 01:57:49.762996 24185 tablet_service.cc:1511] Processing CreateTablet for tablet 81e090d01d8f43cf8cd38e376b7c18ad (DEFAULT_TABLE table=default.table_57697192c9bf44969c2b200e94865a3e [id=ef2e836fffc64aa6a8ddc0425f39e317]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:49.763641 24185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81e090d01d8f43cf8cd38e376b7c18ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:49.763726 26058 raft_consensus.cc:359] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:49.763927 26058 raft_consensus.cc:385] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:49.764025 26058 raft_consensus.cc:740] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50d44445234b4931aaa28141cdaa67ab, State: Initialized, Role: FOLLOWER
I20260810 01:57:49.764331 26058 consensus_queue.cc:260] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:49.767350 26055 ts_tablet_manager.cc:1929] T 4579a90be0b6409781fd9e470324192c P 498d352dbea9405199a121e84a4ab926: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 01:57:49.767520 26055 log.cc:1199] T 4579a90be0b6409781fd9e470324192c P 498d352dbea9405199a121e84a4ab926: Deleting WAL directory at /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-2/wal/wals/4579a90be0b6409781fd9e470324192c
I20260810 01:57:49.769785 26058 ts_tablet_manager.cc:1434] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab: Time spent starting tablet: real 0.007s user 0.000s sys 0.002s
I20260810 01:57:49.771678 26055 ts_tablet_manager.cc:1950] T 4579a90be0b6409781fd9e470324192c P 498d352dbea9405199a121e84a4ab926: Deleting consensus metadata
I20260810 01:57:49.772024 24186 tablet_service.cc:1511] Processing CreateTablet for tablet 5974ae70170544f797ff749280b3be0c (DEFAULT_TABLE table=default.table_983bf2e604df4fdbac75c5aca292f736 [id=552771dde220474a9be9abeefd3a0472]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:49.772552 24186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5974ae70170544f797ff749280b3be0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:49.773536 26058 tablet_bootstrap.cc:492] T 81e090d01d8f43cf8cd38e376b7c18ad P 50d44445234b4931aaa28141cdaa67ab: Bootstrap starting.
I20260810 01:57:49.776110 26058 tablet_bootstrap.cc:654] T 81e090d01d8f43cf8cd38e376b7c18ad P 50d44445234b4931aaa28141cdaa67ab: Neither blocks nor log segments found. Creating new log.
@ 0x7f88b49ca61f clone at ??:0
I20260810 01:57:49.783521 26070 tablet_bootstrap.cc:492] T 5974ae70170544f797ff749280b3be0c P 498d352dbea9405199a121e84a4ab926: Bootstrap starting.
I20260810 01:57:49.786088 26070 tablet_bootstrap.cc:654] T 5974ae70170544f797ff749280b3be0c P 498d352dbea9405199a121e84a4ab926: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:49.787878 26058 tablet_bootstrap.cc:492] T 81e090d01d8f43cf8cd38e376b7c18ad P 50d44445234b4931aaa28141cdaa67ab: No bootstrap required, opened a new log
I20260810 01:57:49.788055 26058 ts_tablet_manager.cc:1403] T 81e090d01d8f43cf8cd38e376b7c18ad P 50d44445234b4931aaa28141cdaa67ab: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.002s
I20260810 01:57:49.789031 26058 raft_consensus.cc:359] T 81e090d01d8f43cf8cd38e376b7c18ad P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:49.789247 26058 raft_consensus.cc:385] T 81e090d01d8f43cf8cd38e376b7c18ad P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:49.789391 26058 raft_consensus.cc:740] T 81e090d01d8f43cf8cd38e376b7c18ad P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50d44445234b4931aaa28141cdaa67ab, State: Initialized, Role: FOLLOWER
I20260810 01:57:49.789749 26058 consensus_queue.cc:260] T 81e090d01d8f43cf8cd38e376b7c18ad P 50d44445234b4931aaa28141cdaa67ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:49.792961 26051 ts_tablet_manager.cc:1950] T 4579a90be0b6409781fd9e470324192c P 48e33bc1e8e94d4887bb64214d0b6c36: Deleting consensus metadata
I20260810 01:57:49.794039 26058 ts_tablet_manager.cc:1434] T 81e090d01d8f43cf8cd38e376b7c18ad P 50d44445234b4931aaa28141cdaa67ab: Time spent starting tablet: real 0.006s user 0.002s sys 0.001s
I20260810 01:57:49.795034 26053 tablet_replica.cc:333] T a48e9f780b4e49abafaadbd78ab8711a P 50d44445234b4931aaa28141cdaa67ab: stopping tablet replica
I20260810 01:57:49.795264 26053 raft_consensus.cc:2243] T a48e9f780b4e49abafaadbd78ab8711a P 50d44445234b4931aaa28141cdaa67ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 01:57:49.795517 26053 raft_consensus.cc:2272] T a48e9f780b4e49abafaadbd78ab8711a P 50d44445234b4931aaa28141cdaa67ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:57:49.796072 26053 ts_tablet_manager.cc:1916] T a48e9f780b4e49abafaadbd78ab8711a P 50d44445234b4931aaa28141cdaa67ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:57:49.798211 26058 tablet_bootstrap.cc:492] T 5974ae70170544f797ff749280b3be0c P 50d44445234b4931aaa28141cdaa67ab: Bootstrap starting.
I20260810 01:57:49.800801 26058 tablet_bootstrap.cc:654] T 5974ae70170544f797ff749280b3be0c P 50d44445234b4931aaa28141cdaa67ab: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:49.801924 26053 ts_tablet_manager.cc:1929] T a48e9f780b4e49abafaadbd78ab8711a P 50d44445234b4931aaa28141cdaa67ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 01:57:49.802070 26053 log.cc:1199] T a48e9f780b4e49abafaadbd78ab8711a P 50d44445234b4931aaa28141cdaa67ab: Deleting WAL directory at /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-0/wal/wals/a48e9f780b4e49abafaadbd78ab8711a
I20260810 01:57:49.802673 26053 ts_tablet_manager.cc:1950] T a48e9f780b4e49abafaadbd78ab8711a P 50d44445234b4931aaa28141cdaa67ab: Deleting consensus metadata
I20260810 01:57:49.802951 26059 tablet_bootstrap.cc:492] T 5974ae70170544f797ff749280b3be0c P 48e33bc1e8e94d4887bb64214d0b6c36: No bootstrap required, opened a new log
I20260810 01:57:49.803139 26059 ts_tablet_manager.cc:1403] T 5974ae70170544f797ff749280b3be0c P 48e33bc1e8e94d4887bb64214d0b6c36: Time spent bootstrapping tablet: real 0.051s user 0.005s sys 0.009s
I20260810 01:57:49.804152 26053 tablet_replica.cc:333] T 4579a90be0b6409781fd9e470324192c P 50d44445234b4931aaa28141cdaa67ab: stopping tablet replica
I20260810 01:57:49.804140 26059 raft_consensus.cc:359] T 5974ae70170544f797ff749280b3be0c P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:49.804423 26053 raft_consensus.cc:2243] T 4579a90be0b6409781fd9e470324192c P 50d44445234b4931aaa28141cdaa67ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 01:57:49.804447 26059 raft_consensus.cc:385] T 5974ae70170544f797ff749280b3be0c P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:49.804608 26059 raft_consensus.cc:740] T 5974ae70170544f797ff749280b3be0c P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48e33bc1e8e94d4887bb64214d0b6c36, State: Initialized, Role: FOLLOWER
I20260810 01:57:49.804623 26053 raft_consensus.cc:2272] T 4579a90be0b6409781fd9e470324192c P 50d44445234b4931aaa28141cdaa67ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:57:49.805363 26053 ts_tablet_manager.cc:1916] T 4579a90be0b6409781fd9e470324192c P 50d44445234b4931aaa28141cdaa67ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:57:49.815588 26059 consensus_queue.cc:260] T 5974ae70170544f797ff749280b3be0c P 48e33bc1e8e94d4887bb64214d0b6c36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:49.815047 26053 ts_tablet_manager.cc:1929] T 4579a90be0b6409781fd9e470324192c P 50d44445234b4931aaa28141cdaa67ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 01:57:49.816614 26053 log.cc:1199] T 4579a90be0b6409781fd9e470324192c P 50d44445234b4931aaa28141cdaa67ab: Deleting WAL directory at /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-0/wal/wals/4579a90be0b6409781fd9e470324192c
I20260810 01:57:49.817124 26053 ts_tablet_manager.cc:1950] T 4579a90be0b6409781fd9e470324192c P 50d44445234b4931aaa28141cdaa67ab: Deleting consensus metadata
I20260810 01:57:49.816398 26058 tablet_bootstrap.cc:492] T 5974ae70170544f797ff749280b3be0c P 50d44445234b4931aaa28141cdaa67ab: No bootstrap required, opened a new log
I20260810 01:57:49.818342 26058 ts_tablet_manager.cc:1403] T 5974ae70170544f797ff749280b3be0c P 50d44445234b4931aaa28141cdaa67ab: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.012s
I20260810 01:57:49.818470 26059 ts_tablet_manager.cc:1434] T 5974ae70170544f797ff749280b3be0c P 48e33bc1e8e94d4887bb64214d0b6c36: Time spent starting tablet: real 0.015s user 0.002s sys 0.000s
I20260810 01:57:49.819387 26058 raft_consensus.cc:359] T 5974ae70170544f797ff749280b3be0c P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:49.819622 26058 raft_consensus.cc:385] T 5974ae70170544f797ff749280b3be0c P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:49.819734 26058 raft_consensus.cc:740] T 5974ae70170544f797ff749280b3be0c P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50d44445234b4931aaa28141cdaa67ab, State: Initialized, Role: FOLLOWER
I20260810 01:57:49.819823 26059 tablet_bootstrap.cc:492] T 81e090d01d8f43cf8cd38e376b7c18ad P 48e33bc1e8e94d4887bb64214d0b6c36: Bootstrap starting.
I20260810 01:57:49.820056 26058 consensus_queue.cc:260] T 5974ae70170544f797ff749280b3be0c P 50d44445234b4931aaa28141cdaa67ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:49.820858 26058 ts_tablet_manager.cc:1434] T 5974ae70170544f797ff749280b3be0c P 50d44445234b4931aaa28141cdaa67ab: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 01:57:49.822067 26059 tablet_bootstrap.cc:654] T 81e090d01d8f43cf8cd38e376b7c18ad P 48e33bc1e8e94d4887bb64214d0b6c36: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:49.836514 26070 tablet_bootstrap.cc:492] T 5974ae70170544f797ff749280b3be0c P 498d352dbea9405199a121e84a4ab926: No bootstrap required, opened a new log
I20260810 01:57:49.836671 26070 ts_tablet_manager.cc:1403] T 5974ae70170544f797ff749280b3be0c P 498d352dbea9405199a121e84a4ab926: Time spent bootstrapping tablet: real 0.053s user 0.003s sys 0.007s
I20260810 01:57:49.837550 26070 raft_consensus.cc:359] T 5974ae70170544f797ff749280b3be0c P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:49.837754 26070 raft_consensus.cc:385] T 5974ae70170544f797ff749280b3be0c P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:49.837852 26070 raft_consensus.cc:740] T 5974ae70170544f797ff749280b3be0c P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 498d352dbea9405199a121e84a4ab926, State: Initialized, Role: FOLLOWER
I20260810 01:57:49.838093 26070 consensus_queue.cc:260] T 5974ae70170544f797ff749280b3be0c P 498d352dbea9405199a121e84a4ab926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:49.838672 26070 ts_tablet_manager.cc:1434] T 5974ae70170544f797ff749280b3be0c P 498d352dbea9405199a121e84a4ab926: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260810 01:57:49.839046 26070 tablet_bootstrap.cc:492] T 81e090d01d8f43cf8cd38e376b7c18ad P 498d352dbea9405199a121e84a4ab926: Bootstrap starting.
I20260810 01:57:49.841216 26070 tablet_bootstrap.cc:654] T 81e090d01d8f43cf8cd38e376b7c18ad P 498d352dbea9405199a121e84a4ab926: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:49.842181 26059 tablet_bootstrap.cc:492] T 81e090d01d8f43cf8cd38e376b7c18ad P 48e33bc1e8e94d4887bb64214d0b6c36: No bootstrap required, opened a new log
I20260810 01:57:49.842360 26059 ts_tablet_manager.cc:1403] T 81e090d01d8f43cf8cd38e376b7c18ad P 48e33bc1e8e94d4887bb64214d0b6c36: Time spent bootstrapping tablet: real 0.023s user 0.006s sys 0.000s
I20260810 01:57:49.843137 26059 raft_consensus.cc:359] T 81e090d01d8f43cf8cd38e376b7c18ad P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:49.843333 26070 tablet_bootstrap.cc:492] T 81e090d01d8f43cf8cd38e376b7c18ad P 498d352dbea9405199a121e84a4ab926: No bootstrap required, opened a new log
I20260810 01:57:49.843343 26059 raft_consensus.cc:385] T 81e090d01d8f43cf8cd38e376b7c18ad P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:49.843452 26059 raft_consensus.cc:740] T 81e090d01d8f43cf8cd38e376b7c18ad P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48e33bc1e8e94d4887bb64214d0b6c36, State: Initialized, Role: FOLLOWER
I20260810 01:57:49.843482 26070 ts_tablet_manager.cc:1403] T 81e090d01d8f43cf8cd38e376b7c18ad P 498d352dbea9405199a121e84a4ab926: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260810 01:57:49.843767 26059 consensus_queue.cc:260] T 81e090d01d8f43cf8cd38e376b7c18ad P 48e33bc1e8e94d4887bb64214d0b6c36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:49.844970 26070 raft_consensus.cc:359] T 81e090d01d8f43cf8cd38e376b7c18ad P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:49.845172 26070 raft_consensus.cc:385] T 81e090d01d8f43cf8cd38e376b7c18ad P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:49.845270 26070 raft_consensus.cc:740] T 81e090d01d8f43cf8cd38e376b7c18ad P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 498d352dbea9405199a121e84a4ab926, State: Initialized, Role: FOLLOWER
I20260810 01:57:49.845378 26059 ts_tablet_manager.cc:1434] T 81e090d01d8f43cf8cd38e376b7c18ad P 48e33bc1e8e94d4887bb64214d0b6c36: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 01:57:49.845561 26070 consensus_queue.cc:260] T 81e090d01d8f43cf8cd38e376b7c18ad P 498d352dbea9405199a121e84a4ab926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:49.846311 26070 ts_tablet_manager.cc:1434] T 81e090d01d8f43cf8cd38e376b7c18ad P 498d352dbea9405199a121e84a4ab926: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 01:57:49.849380 26059 tablet_bootstrap.cc:492] T f88158dec2a449c2a2011b8fc042f0ac P 48e33bc1e8e94d4887bb64214d0b6c36: Bootstrap starting.
I20260810 01:57:49.851572 26059 tablet_bootstrap.cc:654] T f88158dec2a449c2a2011b8fc042f0ac P 48e33bc1e8e94d4887bb64214d0b6c36: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:49.856040 26059 tablet_bootstrap.cc:492] T f88158dec2a449c2a2011b8fc042f0ac P 48e33bc1e8e94d4887bb64214d0b6c36: No bootstrap required, opened a new log
I20260810 01:57:49.856174 26059 ts_tablet_manager.cc:1403] T f88158dec2a449c2a2011b8fc042f0ac P 48e33bc1e8e94d4887bb64214d0b6c36: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260810 01:57:49.856962 26059 raft_consensus.cc:359] T f88158dec2a449c2a2011b8fc042f0ac P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:49.857132 26059 raft_consensus.cc:385] T f88158dec2a449c2a2011b8fc042f0ac P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:49.857218 26059 raft_consensus.cc:740] T f88158dec2a449c2a2011b8fc042f0ac P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48e33bc1e8e94d4887bb64214d0b6c36, State: Initialized, Role: FOLLOWER
I20260810 01:57:49.857523 26059 consensus_queue.cc:260] T f88158dec2a449c2a2011b8fc042f0ac P 48e33bc1e8e94d4887bb64214d0b6c36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:49.859161 26059 ts_tablet_manager.cc:1434] T f88158dec2a449c2a2011b8fc042f0ac P 48e33bc1e8e94d4887bb64214d0b6c36: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 01:57:49.866348 25814 raft_consensus.cc:493] T 81e090d01d8f43cf8cd38e376b7c18ad P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:57:49.866587 25814 raft_consensus.cc:515] T 81e090d01d8f43cf8cd38e376b7c18ad P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:49.868177 25814 leader_election.cc:290] T 81e090d01d8f43cf8cd38e376b7c18ad P 50d44445234b4931aaa28141cdaa67ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
I20260810 01:57:49.868541 23997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81e090d01d8f43cf8cd38e376b7c18ad" candidate_uuid: "50d44445234b4931aaa28141cdaa67ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" is_pre_election: true
I20260810 01:57:49.868763 23997 raft_consensus.cc:2468] T 81e090d01d8f43cf8cd38e376b7c18ad P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50d44445234b4931aaa28141cdaa67ab in term 0.
I20260810 01:57:49.869004 24222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81e090d01d8f43cf8cd38e376b7c18ad" candidate_uuid: "50d44445234b4931aaa28141cdaa67ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "498d352dbea9405199a121e84a4ab926" is_pre_election: true
I20260810 01:57:49.869270 24222 raft_consensus.cc:2468] T 81e090d01d8f43cf8cd38e376b7c18ad P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50d44445234b4931aaa28141cdaa67ab in term 0.
I20260810 01:57:49.869303 23679 leader_election.cc:304] T 81e090d01d8f43cf8cd38e376b7c18ad P 50d44445234b4931aaa28141cdaa67ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab; no voters:
I20260810 01:57:49.870404 25814 raft_consensus.cc:2804] T 81e090d01d8f43cf8cd38e376b7c18ad P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 01:57:49.870523 25814 raft_consensus.cc:493] T 81e090d01d8f43cf8cd38e376b7c18ad P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 01:57:49.870608 25814 raft_consensus.cc:3060] T 81e090d01d8f43cf8cd38e376b7c18ad P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:49.873812 25814 raft_consensus.cc:515] T 81e090d01d8f43cf8cd38e376b7c18ad P 50d44445234b4931aaa28141cdaa67ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:49.874804 23997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81e090d01d8f43cf8cd38e376b7c18ad" candidate_uuid: "50d44445234b4931aaa28141cdaa67ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48e33bc1e8e94d4887bb64214d0b6c36"
I20260810 01:57:49.874886 24222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81e090d01d8f43cf8cd38e376b7c18ad" candidate_uuid: "50d44445234b4931aaa28141cdaa67ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "498d352dbea9405199a121e84a4ab926"
I20260810 01:57:49.875095 24222 raft_consensus.cc:3060] T 81e090d01d8f43cf8cd38e376b7c18ad P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:49.875056 23997 raft_consensus.cc:3060] T 81e090d01d8f43cf8cd38e376b7c18ad P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:49.874433 25814 leader_election.cc:290] T 81e090d01d8f43cf8cd38e376b7c18ad P 50d44445234b4931aaa28141cdaa67ab [CANDIDATE]: Term 1 election: Requested vote from peers 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
I20260810 01:57:49.878242 24222 raft_consensus.cc:2468] T 81e090d01d8f43cf8cd38e376b7c18ad P 498d352dbea9405199a121e84a4ab926 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50d44445234b4931aaa28141cdaa67ab in term 1.
I20260810 01:57:49.878598 23997 raft_consensus.cc:2468] T 81e090d01d8f43cf8cd38e376b7c18ad P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50d44445234b4931aaa28141cdaa67ab in term 1.
I20260810 01:57:49.878844 23684 leader_election.cc:304] T 81e090d01d8f43cf8cd38e376b7c18ad P 50d44445234b4931aaa28141cdaa67ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab; no voters:
I20260810 01:57:49.879361 25814 raft_consensus.cc:2804] T 81e090d01d8f43cf8cd38e376b7c18ad P 50d44445234b4931aaa28141cdaa67ab [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:57:49.879917 25814 raft_consensus.cc:697] T 81e090d01d8f43cf8cd38e376b7c18ad P 50d44445234b4931aaa28141cdaa67ab [term 1 LEADER]: Becoming Leader. State: Replica: 50d44445234b4931aaa28141cdaa67ab, State: Running, Role: LEADER
I20260810 01:57:49.880290 25814 consensus_queue.cc:237] T 81e090d01d8f43cf8cd38e376b7c18ad P 50d44445234b4931aaa28141cdaa67ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
W20260810 01:57:50.006497 23684 connection.cc:573] server connection from 127.0.0.1:42574 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 01:57:50.006456 23890 connection.cc:573] server connection from 127.0.0.1:43262 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 01:57:50.006456 24118 connection.cc:573] server connection from 127.0.0.1:46950 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 01:57:50.006757 23890 connection.cc:573] client connection to 127.22.69.61:41945 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 01:57:50.006773 24111 connection.cc:573] client connection to 127.22.69.61:41945 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 01:57:50.007470 25864 connection.cc:573] server connection from 127.0.0.1:56144 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 01:57:50.007618 24317 connection.cc:573] client connection to 127.22.69.61:41945 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 01:57:50.008345 24317 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.22.69.61:41945)
W20260810 01:57:50.008860 24317 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.22.69.61:41945)
W20260810 01:57:50.009097 24317 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.22.69.61:41945)
W20260810 01:57:50.009781 24317 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.22.69.61:41945)
I20260810 01:57:50.138228 25601 raft_consensus.cc:493] T 5974ae70170544f797ff749280b3be0c P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:57:50.138406 25601 raft_consensus.cc:515] T 5974ae70170544f797ff749280b3be0c P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:50.139002 25601 leader_election.cc:290] T 5974ae70170544f797ff749280b3be0c P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
I20260810 01:57:50.139469 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5974ae70170544f797ff749280b3be0c" candidate_uuid: "498d352dbea9405199a121e84a4ab926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50d44445234b4931aaa28141cdaa67ab" is_pre_election: true
I20260810 01:57:50.139683 23776 raft_consensus.cc:2468] T 5974ae70170544f797ff749280b3be0c P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 498d352dbea9405199a121e84a4ab926 in term 0.
I20260810 01:57:50.139602 23997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5974ae70170544f797ff749280b3be0c" candidate_uuid: "498d352dbea9405199a121e84a4ab926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" is_pre_election: true
I20260810 01:57:50.139863 23997 raft_consensus.cc:2468] T 5974ae70170544f797ff749280b3be0c P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 498d352dbea9405199a121e84a4ab926 in term 0.
I20260810 01:57:50.140120 24109 leader_election.cc:304] T 5974ae70170544f797ff749280b3be0c P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab; no voters:
I20260810 01:57:50.140379 25601 raft_consensus.cc:2804] T 5974ae70170544f797ff749280b3be0c P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 01:57:50.140501 25601 raft_consensus.cc:493] T 5974ae70170544f797ff749280b3be0c P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 01:57:50.140604 25601 raft_consensus.cc:3060] T 5974ae70170544f797ff749280b3be0c P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:50.142602 25601 raft_consensus.cc:515] T 5974ae70170544f797ff749280b3be0c P 498d352dbea9405199a121e84a4ab926 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:50.143151 25601 leader_election.cc:290] T 5974ae70170544f797ff749280b3be0c P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 election: Requested vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
I20260810 01:57:50.143544 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5974ae70170544f797ff749280b3be0c" candidate_uuid: "498d352dbea9405199a121e84a4ab926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50d44445234b4931aaa28141cdaa67ab"
I20260810 01:57:50.143626 23997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5974ae70170544f797ff749280b3be0c" candidate_uuid: "498d352dbea9405199a121e84a4ab926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48e33bc1e8e94d4887bb64214d0b6c36"
I20260810 01:57:50.143756 23776 raft_consensus.cc:3060] T 5974ae70170544f797ff749280b3be0c P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:50.143802 23997 raft_consensus.cc:3060] T 5974ae70170544f797ff749280b3be0c P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:50.145744 23776 raft_consensus.cc:2468] T 5974ae70170544f797ff749280b3be0c P 50d44445234b4931aaa28141cdaa67ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 498d352dbea9405199a121e84a4ab926 in term 1.
I20260810 01:57:50.145990 23997 raft_consensus.cc:2468] T 5974ae70170544f797ff749280b3be0c P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 498d352dbea9405199a121e84a4ab926 in term 1.
I20260810 01:57:50.146198 24109 leader_election.cc:304] T 5974ae70170544f797ff749280b3be0c P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab; no voters:
I20260810 01:57:50.146521 25601 raft_consensus.cc:2804] T 5974ae70170544f797ff749280b3be0c P 498d352dbea9405199a121e84a4ab926 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:57:50.146936 25601 raft_consensus.cc:697] T 5974ae70170544f797ff749280b3be0c P 498d352dbea9405199a121e84a4ab926 [term 1 LEADER]: Becoming Leader. State: Replica: 498d352dbea9405199a121e84a4ab926, State: Running, Role: LEADER
I20260810 01:57:50.147470 25601 consensus_queue.cc:237] T 5974ae70170544f797ff749280b3be0c P 498d352dbea9405199a121e84a4ab926 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:50.241895 25814 raft_consensus.cc:493] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:57:50.242092 25814 raft_consensus.cc:515] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:50.242568 25601 raft_consensus.cc:493] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:57:50.242713 25814 leader_election.cc:290] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
I20260810 01:57:50.243201 25601 raft_consensus.cc:515] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:50.243296 23997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f88158dec2a449c2a2011b8fc042f0ac" candidate_uuid: "50d44445234b4931aaa28141cdaa67ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" is_pre_election: true
I20260810 01:57:50.243559 23997 raft_consensus.cc:2468] T f88158dec2a449c2a2011b8fc042f0ac P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50d44445234b4931aaa28141cdaa67ab in term 0.
W20260810 01:57:50.243655 23684 leader_election.cc:343] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: f88158dec2a449c2a2011b8fc042f0ac
I20260810 01:57:50.244060 23679 leader_election.cc:304] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab; no voters: 498d352dbea9405199a121e84a4ab926
I20260810 01:57:50.244380 25814 raft_consensus.cc:2804] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 01:57:50.244470 25601 leader_election.cc:290] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
I20260810 01:57:50.244520 25814 raft_consensus.cc:493] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 01:57:50.244645 25814 raft_consensus.cc:3060] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Advancing to term 1
W20260810 01:57:50.244966 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
W20260810 01:57:50.245221 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
I20260810 01:57:50.245328 24109 leader_election.cc:304] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:57:50.245612 25601 raft_consensus.cc:2749] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:57:50.246968 25814 raft_consensus.cc:515] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:50.247633 25814 leader_election.cc:290] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab [CANDIDATE]: Term 1 election: Requested vote from peers 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
I20260810 01:57:50.248091 23997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f88158dec2a449c2a2011b8fc042f0ac" candidate_uuid: "50d44445234b4931aaa28141cdaa67ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48e33bc1e8e94d4887bb64214d0b6c36"
W20260810 01:57:50.248262 23684 leader_election.cc:343] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: f88158dec2a449c2a2011b8fc042f0ac
I20260810 01:57:50.248322 23997 raft_consensus.cc:3060] T f88158dec2a449c2a2011b8fc042f0ac P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:50.250217 23997 raft_consensus.cc:2468] T f88158dec2a449c2a2011b8fc042f0ac P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50d44445234b4931aaa28141cdaa67ab in term 1.
I20260810 01:57:50.250581 23679 leader_election.cc:304] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab; no voters: 498d352dbea9405199a121e84a4ab926
I20260810 01:57:50.250852 25814 raft_consensus.cc:2804] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:57:50.251029 25814 raft_consensus.cc:697] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab [term 1 LEADER]: Becoming Leader. State: Replica: 50d44445234b4931aaa28141cdaa67ab, State: Running, Role: LEADER
I20260810 01:57:50.251480 25814 consensus_queue.cc:237] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:50.261620 26094 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 01:57:50.261780 26094 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:50.263451 26094 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:57:50.264269 25865 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 01:57:50.265722 25865 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 01:57:50.265969 25865 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:57:50.266091 25865 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 01:57:50.266443 26094 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260810 01:57:50.458724 25601 raft_consensus.cc:493] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:57:50.458987 25601 raft_consensus.cc:515] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:50.459666 25601 leader_election.cc:290] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:57:50.460841 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
W20260810 01:57:50.461079 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
I20260810 01:57:50.461205 24109 leader_election.cc:304] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:57:50.461568 25601 raft_consensus.cc:2749] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:57:50.824003 25601 raft_consensus.cc:493] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:57:50.824239 25601 raft_consensus.cc:515] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:50.824908 25601 leader_election.cc:290] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:57:50.825851 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
W20260810 01:57:50.826133 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
I20260810 01:57:50.826238 24109 leader_election.cc:304] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:57:50.826534 25601 raft_consensus.cc:2749] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:57:50.829205 25814 consensus_queue.cc:1048] T 81e090d01d8f43cf8cd38e376b7c18ad P 50d44445234b4931aaa28141cdaa67ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:57:50.837607 25814 consensus_queue.cc:1048] T 81e090d01d8f43cf8cd38e376b7c18ad P 50d44445234b4931aaa28141cdaa67ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:57:50.945051 25601 consensus_queue.cc:1048] T 5974ae70170544f797ff749280b3be0c P 498d352dbea9405199a121e84a4ab926 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:57:50.952235 26128 consensus_queue.cc:1048] T 5974ae70170544f797ff749280b3be0c P 498d352dbea9405199a121e84a4ab926 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:57:50.953294 26129 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 01:57:50.953532 26129 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:50.955418 26129 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:57:50.959988 25865 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 01:57:50.960631 25865 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:57:50.960785 25865 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 01:57:50.961040 26129 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260810 01:57:51.116927 23684 consensus_peers.cc:597] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f88158dec2a449c2a2011b8fc042f0ac. This is attempt 1: this message will repeat every 5th retry.
I20260810 01:57:51.443346 26120 consensus_queue.cc:1048] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:57:51.770025 26154 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 01:57:51.770246 26154 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:51.770831 26154 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:57:51.773049 25865 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 01:57:51.773283 25865 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:57:51.773424 25865 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 01:57:51.773686 26154 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260810 01:57:52.007167 22804 external_mini_cluster.cc:1372] Running /tmp/dist-test-task_qhVVJ/build/asan/bin/kudu
/tmp/dist-test-task_qhVVJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.69.61:41945
--webserver_interface=127.22.69.61
--webserver_port=33591
--builtin_ntp_servers=127.22.69.20:43209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917
--hive_metastore_uris=thrift://127.0.0.1:37513
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260810 01:57:52.139595 26159 ts_tablet_manager.cc:933] T f88158dec2a449c2a2011b8fc042f0ac P 498d352dbea9405199a121e84a4ab926: Initiating tablet copy from peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491)
I20260810 01:57:52.140625 26159 tablet_copy_client.cc:323] T f88158dec2a449c2a2011b8fc042f0ac P 498d352dbea9405199a121e84a4ab926: tablet copy: Beginning tablet copy session from remote peer at address 127.22.69.1:45491
I20260810 01:57:52.141522 23810 tablet_copy_service.cc:140] P 50d44445234b4931aaa28141cdaa67ab: Received BeginTabletCopySession request for tablet f88158dec2a449c2a2011b8fc042f0ac from peer 498d352dbea9405199a121e84a4ab926 ({username='slave'} at 127.22.69.3:34189)
I20260810 01:57:52.141708 23810 tablet_copy_service.cc:161] P 50d44445234b4931aaa28141cdaa67ab: Beginning new tablet copy session on tablet f88158dec2a449c2a2011b8fc042f0ac from peer 498d352dbea9405199a121e84a4ab926 at {username='slave'} at 127.22.69.3:34189: session id = 498d352dbea9405199a121e84a4ab926-f88158dec2a449c2a2011b8fc042f0ac
I20260810 01:57:52.143839 23810 tablet_copy_source_session.cc:215] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 01:57:52.145488 26159 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f88158dec2a449c2a2011b8fc042f0ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:52.151379 26159 tablet_copy_client.cc:806] T f88158dec2a449c2a2011b8fc042f0ac P 498d352dbea9405199a121e84a4ab926: tablet copy: Starting download of 0 data blocks...
I20260810 01:57:52.151767 26159 tablet_copy_client.cc:670] T f88158dec2a449c2a2011b8fc042f0ac P 498d352dbea9405199a121e84a4ab926: tablet copy: Starting download of 1 WAL segments...
I20260810 01:57:52.153812 26159 tablet_copy_client.cc:538] T f88158dec2a449c2a2011b8fc042f0ac P 498d352dbea9405199a121e84a4ab926: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 01:57:52.156601 26159 tablet_bootstrap.cc:492] T f88158dec2a449c2a2011b8fc042f0ac P 498d352dbea9405199a121e84a4ab926: Bootstrap starting.
I20260810 01:57:52.161911 26159 tablet_bootstrap.cc:492] T f88158dec2a449c2a2011b8fc042f0ac P 498d352dbea9405199a121e84a4ab926: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 01:57:52.162334 26159 tablet_bootstrap.cc:492] T f88158dec2a449c2a2011b8fc042f0ac P 498d352dbea9405199a121e84a4ab926: Bootstrap complete.
I20260810 01:57:52.162580 26159 ts_tablet_manager.cc:1403] T f88158dec2a449c2a2011b8fc042f0ac P 498d352dbea9405199a121e84a4ab926: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260810 01:57:52.163324 26159 raft_consensus.cc:359] T f88158dec2a449c2a2011b8fc042f0ac P 498d352dbea9405199a121e84a4ab926 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:52.163529 26159 raft_consensus.cc:740] T f88158dec2a449c2a2011b8fc042f0ac P 498d352dbea9405199a121e84a4ab926 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 498d352dbea9405199a121e84a4ab926, State: Initialized, Role: FOLLOWER
I20260810 01:57:52.163786 26159 consensus_queue.cc:260] T f88158dec2a449c2a2011b8fc042f0ac P 498d352dbea9405199a121e84a4ab926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:52.164353 26159 ts_tablet_manager.cc:1434] T f88158dec2a449c2a2011b8fc042f0ac P 498d352dbea9405199a121e84a4ab926: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 01:57:52.165884 23810 tablet_copy_service.cc:342] P 50d44445234b4931aaa28141cdaa67ab: Request end of tablet copy session 498d352dbea9405199a121e84a4ab926-f88158dec2a449c2a2011b8fc042f0ac received from {username='slave'} at 127.22.69.3:34189
I20260810 01:57:52.166023 23810 tablet_copy_service.cc:434] P 50d44445234b4931aaa28141cdaa67ab: ending tablet copy session 498d352dbea9405199a121e84a4ab926-f88158dec2a449c2a2011b8fc042f0ac on tablet f88158dec2a449c2a2011b8fc042f0ac with peer 498d352dbea9405199a121e84a4ab926
W20260810 01:57:52.353626 26158 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:57:52.354194 26158 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:57:52.354377 26158 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260810 01:57:52.363741 26158 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:57:52.363819 26158 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:57:52.363874 26158 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:57:52.363922 26158 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:57:52.373687 26158 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.69.20:43209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:37513
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.69.61:41945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.69.61
--webserver_port=33591
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 10 Aug 2026 01:44:04 UTC on e7f111948823
build id 13266
ASAN enabled
I20260810 01:57:52.375231 26158 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:57:52.376952 26158 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260810 01:57:52.384627 26185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:57:52.385373 26184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:57:52.385421 26187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:57:52.386137 26158 server_base.cc:1061] running on GCE node
I20260810 01:57:52.387105 26158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:57:52.388592 26158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:57:52.389812 26158 hybrid_clock.cc:648] HybridClock initialized: now 1786327072389757 us; error 33 us; skew 500 ppm
I20260810 01:57:52.390166 26158 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 01:57:52.392275 26158 webserver.cc:533] Webserver started at http://127.22.69.61:33591/ using document root <none> and password file <none>
I20260810 01:57:52.393002 26158 fs_manager.cc:362] Metadata directory not provided
I20260810 01:57:52.393141 26158 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:57:52.398592 26158 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 01:57:52.401494 26195 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:57:52.402886 26158 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 01:57:52.403110 26158 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/data,/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/wal
uuid: "b1422349180c446db6c1768b3025253c"
format_stamp: "Formatted at 2026-08-10 01:57:26 on dist-test-slave-n326"
I20260810 01:57:52.403887 26158 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:57:52.422685 26158 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260810 01:57:52.424046 26158 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:57:52.424413 26158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:57:52.444787 26158 rpc_server.cc:307] RPC server started. Bound to: 127.22.69.61:41945
I20260810 01:57:52.444808 26282 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.22.69.61:41945 every 8 connection(s)
I20260810 01:57:52.446410 26158 server_base.cc:1193] Dumped server information to /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-1/data/info.pb
I20260810 01:57:52.449417 22804 external_mini_cluster.cc:1434] Started /tmp/dist-test-task_qhVVJ/build/asan/bin/kudu as pid 26158
I20260810 01:57:52.454557 26284 sys_catalog.cc:263] Verifying existing consensus state
I20260810 01:57:52.457931 26284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c: Bootstrap starting.
I20260810 01:57:52.470191 26284 log.cc:826] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c: Log is configured to *not* fsync() on all Append() calls
I20260810 01:57:52.484730 26292 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 01:57:52.484913 26292 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:52.485692 26292 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:57:52.495725 25865 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:57:52.512065 26284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c: Bootstrap replayed 1/1 log segments. Stats: ops{read=46 overwritten=0 applied=46 ignored=0} inserts{seen=31 ignored=0} mutations{seen=33 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 01:57:52.512648 26284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c: Bootstrap complete.
I20260810 01:57:52.520311 26284 raft_consensus.cc:359] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 7 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:52.521262 26284 raft_consensus.cc:740] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 7 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1422349180c446db6c1768b3025253c, State: Initialized, Role: FOLLOWER
I20260810 01:57:52.521989 26284 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 46, Last appended: 7.46, Last appended by leader: 46, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:52.522795 26297 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 7 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:52.523092 26297 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:57:52.524185 26284 sys_catalog.cc:565] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:57:52.524245 26243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "919313c0dee540a690f46dbc8208d20b" candidate_term: 8 candidate_status { last_received { term: 7 index: 46 } } ignore_live_leader: false dest_uuid: "b1422349180c446db6c1768b3025253c" is_pre_election: true
I20260810 01:57:52.524796 26243 raft_consensus.cc:2468] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 919313c0dee540a690f46dbc8208d20b in term 7.
I20260810 01:57:52.525439 25865 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b, b1422349180c446db6c1768b3025253c; no voters: 757bb3f336084510845ed58df264c721
I20260810 01:57:52.526098 26292 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260810 01:57:52.526237 26292 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 7 FOLLOWER]: Starting leader election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 01:57:52.526362 26292 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 7 FOLLOWER]: Advancing to term 8
I20260810 01:57:52.530081 26292 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:52.530938 26292 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 8 election: Requested vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
I20260810 01:57:52.531672 26243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "919313c0dee540a690f46dbc8208d20b" candidate_term: 8 candidate_status { last_received { term: 7 index: 46 } } ignore_live_leader: false dest_uuid: "b1422349180c446db6c1768b3025253c"
I20260810 01:57:52.531913 26243 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 7 FOLLOWER]: Advancing to term 8
W20260810 01:57:52.533465 25865 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:57:52.537743 26243 raft_consensus.cc:2468] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 919313c0dee540a690f46dbc8208d20b in term 8.
I20260810 01:57:52.538463 25865 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b, b1422349180c446db6c1768b3025253c; no voters: 757bb3f336084510845ed58df264c721
I20260810 01:57:52.539988 26292 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 8 FOLLOWER]: Leader election won for term 8
I20260810 01:57:52.554102 26322 catalog_manager.cc:1295] Loaded cluster ID: dfa48c56b63b4d0cbf3d6124921e8a17
I20260810 01:57:52.554284 26322 catalog_manager.cc:1588] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c: loading cluster ID for follower catalog manager: success
I20260810 01:57:52.555667 26292 raft_consensus.cc:697] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 8 LEADER]: Becoming Leader. State: Replica: 919313c0dee540a690f46dbc8208d20b, State: Running, Role: LEADER
I20260810 01:57:52.556457 26292 consensus_queue.cc:237] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 46, Committed index: 46, Last appended: 7.46, Last appended by leader: 46, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:52.559806 26322 catalog_manager.cc:1610] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c: acquiring CA information for follower catalog manager: success
I20260810 01:57:52.571672 26322 catalog_manager.cc:1638] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 01:57:52.571926 26330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 919313c0dee540a690f46dbc8208d20b. Latest consensus state: current_term: 8 leader_uuid: "919313c0dee540a690f46dbc8208d20b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:52.572455 26330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: This master's current role is: LEADER
I20260810 01:57:52.577111 26336 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260810 01:57:52.581512 26336 catalog_manager.cc:679] Loaded metadata for table default.table_72b6f93a124042e4a090dc194134c75a [id=017756566b72415aa33842f8df2b3e91]
I20260810 01:57:52.582060 26336 catalog_manager.cc:679] Loaded metadata for table default.table_73d521a2c5504815ad6d7cc3dd0bf818 [id=2f1832807601444a88d61b8396a67db9]
I20260810 01:57:52.582643 26336 catalog_manager.cc:679] Loaded metadata for table default.table_c8ad906be0f74210912a57adead21255 [id=4da3c920ea6a48588c9f0c91da00fb69]
I20260810 01:57:52.583165 26336 catalog_manager.cc:679] Loaded metadata for table default.table_983bf2e604df4fdbac75c5aca292f736 [id=552771dde220474a9be9abeefd3a0472]
I20260810 01:57:52.583668 26336 catalog_manager.cc:679] Loaded metadata for table default.table_22726fd1a11f409f9f5287ab16d8e551 [id=5edf9cb336414ae3bd41e9f5ca41fcca]
I20260810 01:57:52.584156 26336 catalog_manager.cc:679] Loaded metadata for table default.table_0ba2046f969346f68f1bde7cf6c9f273 [id=630b03060b7646118927a18573c0920c]
I20260810 01:57:52.584619 26336 catalog_manager.cc:679] Loaded metadata for table default.table_510ae1b5fcc74db788ffb64925cad250 [id=7d240d10089c47eaa014d59c4ac7f0b4]
I20260810 01:57:52.585182 26336 catalog_manager.cc:679] Loaded metadata for table default.table_379ea52611ef49d591a022addb7afc0c [id=87ba140e9080402aa494198d6f212f05]
I20260810 01:57:52.585669 26336 catalog_manager.cc:679] Loaded metadata for table default.table_57697192c9bf44969c2b200e94865a3e [id=ef2e836fffc64aa6a8ddc0425f39e317]
I20260810 01:57:52.588798 26336 tablet_loader.cc:96] loaded metadata for tablet 1c3db8429f9f4cf5b29d3d1a6cc0651c (table default.table_379ea52611ef49d591a022addb7afc0c [id=87ba140e9080402aa494198d6f212f05])
I20260810 01:57:52.589152 26336 tablet_loader.cc:96] loaded metadata for tablet 2230d919953e49208659450a024c6a62 (table default.table_0ba2046f969346f68f1bde7cf6c9f273 [id=630b03060b7646118927a18573c0920c])
I20260810 01:57:52.589510 26336 tablet_loader.cc:96] loaded metadata for tablet 4514cc930ba44e0f92a438f8ae0ba6fc (table default.table_510ae1b5fcc74db788ffb64925cad250 [id=7d240d10089c47eaa014d59c4ac7f0b4])
I20260810 01:57:52.589998 26336 tablet_loader.cc:96] loaded metadata for tablet 5974ae70170544f797ff749280b3be0c (table default.table_983bf2e604df4fdbac75c5aca292f736 [id=552771dde220474a9be9abeefd3a0472])
I20260810 01:57:52.590335 26336 tablet_loader.cc:96] loaded metadata for tablet 5c49a207b37e40579d34002c3fb57afb (table default.table_c8ad906be0f74210912a57adead21255 [id=4da3c920ea6a48588c9f0c91da00fb69])
I20260810 01:57:52.590649 26336 tablet_loader.cc:96] loaded metadata for tablet 6ca335f1e5964f8490a49ba4c4fb4c72 (table default.table_72b6f93a124042e4a090dc194134c75a [id=017756566b72415aa33842f8df2b3e91])
I20260810 01:57:52.590969 26336 tablet_loader.cc:96] loaded metadata for tablet 81e090d01d8f43cf8cd38e376b7c18ad (table default.table_57697192c9bf44969c2b200e94865a3e [id=ef2e836fffc64aa6a8ddc0425f39e317])
I20260810 01:57:52.591933 26336 tablet_loader.cc:96] loaded metadata for tablet cad2404759604bfea7c273e1aa40539a (table default.table_73d521a2c5504815ad6d7cc3dd0bf818 [id=2f1832807601444a88d61b8396a67db9])
I20260810 01:57:52.592447 26336 tablet_loader.cc:96] loaded metadata for tablet f88158dec2a449c2a2011b8fc042f0ac (table default.table_22726fd1a11f409f9f5287ab16d8e551 [id=5edf9cb336414ae3bd41e9f5ca41fcca])
I20260810 01:57:52.592727 26336 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260810 01:57:52.593780 26336 catalog_manager.cc:1295] Loaded cluster ID: dfa48c56b63b4d0cbf3d6124921e8a17
I20260810 01:57:52.593892 26336 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260810 01:57:52.595517 26336 catalog_manager.cc:1540] Loading token signing keys...
I20260810 01:57:52.596742 26336 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b: Loaded TSK: 0
I20260810 01:57:52.597697 26336 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260810 01:57:52.598814 26336 catalog_manager.cc:1563] Loading latest processed Hive Metastore notification log event ID...
I20260810 01:57:52.600010 26336 catalog_manager.cc:5941] Last processed Hive Metastore notification event ID: -1
I20260810 01:57:52.604475 26158 master_runner.cc:187] Error getting master registration for 127.22.69.60:36917: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111),
I20260810 01:57:52.608446 26158 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 01:57:52.647496 25889 master_service.cc:960] ReplaceTablet: received request to replace tablet 09ed68395e2d44a5b51a22abe2203406 from {username='slave'} at 127.0.0.1:56142
I20260810 01:57:52.648590 25878 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56142:
table { table_name: "default.Table_510ae1b5fcc74db788ffb64925cad250" } modify_external_catalogs: true
I20260810 01:57:52.648929 25878 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56142:
table { table_name: "default.Table_510ae1b5fcc74db788ffb64925cad250" } modify_external_catalogs: true
I20260810 01:57:52.648993 25874 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56142:
table { table_name: "default.Table_0ba2046f969346f68f1bde7cf6c9f273" } modify_external_catalogs: true
I20260810 01:57:52.649276 25874 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56142:
table { table_name: "default.Table_0ba2046f969346f68f1bde7cf6c9f273" } modify_external_catalogs: true
I20260810 01:57:52.649382 25885 master_service.cc:960] ReplaceTablet: received request to replace tablet 81e090d01d8f43cf8cd38e376b7c18ad from {username='slave'} at 127.0.0.1:56142
I20260810 01:57:52.650784 25888 master_service.cc:960] ReplaceTablet: received request to replace tablet 09ed68395e2d44a5b51a22abe2203406 from {username='slave'} at 127.0.0.1:56142
I20260810 01:57:52.650666 25876 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56142:
name: "default.Table_7e5efb367a02411aac568a75dd39b274"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:52.651901 25883 master_service.cc:960] ReplaceTablet: received request to replace tablet 1844a63489d74f1c919e96501e4bd568 from {username='slave'} at 127.0.0.1:56142
I20260810 01:57:52.651998 25889 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56142:
name: "default.Table_14c5bada100e48adb5f378e73ee7d00b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:52.651312 25882 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56142:
name: "default.Table_1bd23f5807da48238cbddc7597b8ab50"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:52.653947 25875 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56142:
table { table_name: "default.Table_57697192c9bf44969c2b200e94865a3e" } modify_external_catalogs: true
I20260810 01:57:52.654147 25875 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56142:
table { table_name: "default.Table_57697192c9bf44969c2b200e94865a3e" } modify_external_catalogs: true
I20260810 01:57:52.653328 25880 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56142:
name: "default.Table_bb2d4e224bd14be09d60bce7977d4c19"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:52.654627 25883 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56142:
name: "default.Table_9175c2bd9b0542f2a6b95eae8a882b7a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:52.657140 25888 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56142:
name: "default.Table_b03802d8f2f14a24b00ab6efbe9392dc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260810 01:57:52.659888 25989 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 10 received events: NotificationEvent(eventId=1, eventTime=1786327049, eventType=CREATE_TABLE, dbName=default, tableName=table_4fb63c687c11493ca7c0b76735f137e9, message={"server":"","servicePrincipal":"","db":"default","table":"table_4fb63c687c11493ca7c0b76735f137e9","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_4fb63c687c11493ca7c0b76735f137e9\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1786327049},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/warehouse/table_4fb63c687c11493ca7c0b76735f137e9\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_4fb63c687c11493ca7c0b76735f137e9\",\"kudu.table_id\":\"cb924727e6014a2a9dd271f962df3807\",\"transient_lastDdlTime\":\"1786327049\",\"comment\":\"\",\"kudu.master_addresses\":\"127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"dfa48c56b63b4d0cbf3d6124921e8a17\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1786327049,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1786327049, eventType=CREATE_TABLE, dbName=default, tableName=table_c8ad906be0f74210912a57adead21255, message={"server":"","servicePrincipal":"","db":"default","table":"table_c8ad906be0f74210912a57adead21255","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_c8ad906be0f74210912a57adead21255\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1786327049},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/warehouse/table_c8ad906be0f74210912a57adead21255\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_c8ad906be0f74210912a57adead21255\",\"kudu.table_id\":\"4da3c920ea6a48588c9f0c91da00fb69\",\"transient_lastDdlTime\":\"1786327049\",\"comment\":\"\",\"kudu.master_addresses\":\"127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"dfa48c56b63b4d0cbf3d6124921e8a17\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1786327049,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
I20260810 01:57:52.661221 26243 raft_consensus.cc:1275] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 8 FOLLOWER]: Refusing update from remote peer 919313c0dee540a690f46dbc8208d20b: Log matching property violated. Preceding OpId in replica: term: 7 index: 46. Preceding OpId from leader: term: 8 index: 48. (index mismatch)
W20260810 01:57:52.661698 25865 consensus_peers.cc:597] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b -> Peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Couldn't send request to peer 757bb3f336084510845ed58df264c721. Status: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 01:57:52.662184 26330 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 47, Last known committed idx: 46, Time since last communication: 0.000s
@ 0x7f6b3fb45d5d google::LogMessage::Fail() at ??:0
@ 0x7f6b3fb49b23 google::LogMessage::SendToLog() at ??:0
@ 0x7f6b3fb4575c google::LogMessage::Flush() at ??:0
@ 0x7f6b3fb459a1 google::LogMessage::~LogMessage() at ??:0
I20260810 01:57:52.668263 26297 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 919313c0dee540a690f46dbc8208d20b. Latest consensus state: current_term: 8 leader_uuid: "919313c0dee540a690f46dbc8208d20b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:52.668521 26297 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:57:52.680075 24301 heartbeater.cc:344] Connected to a master server at 127.22.69.61:41945
@ 0x7f6b5680ef64 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260810 01:57:52.687440 26292 sys_catalog.cc:455] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 8 leader_uuid: "919313c0dee540a690f46dbc8208d20b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:52.687762 26292 sys_catalog.cc:458] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: This master's current role is: LEADER
I20260810 01:57:52.689132 25885 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b: Sending DeleteTablet for 3 replicas of tablet 81e090d01d8f43cf8cd38e376b7c18ad
I20260810 01:57:52.691282 25885 catalog_manager.cc:6807] ReplaceTablet: tablet 81e090d01d8f43cf8cd38e376b7c18ad deleted and replaced by tablet 9b595b2ec2ff4503a20c1e9e7a7467b7
I20260810 01:57:52.692365 26229 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "498d352dbea9405199a121e84a4ab926" instance_seqno: 1786327049319804) as {username='slave'} at 127.22.69.3:55115; Asking this server to re-register.
I20260810 01:57:52.693470 24301 heartbeater.cc:461] Registering TS with master...
@ 0x7f6b5680d811 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260810 01:57:52.700117 26297 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 8 leader_uuid: "919313c0dee540a690f46dbc8208d20b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:52.700328 26297 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:57:52.700973 26229 ts_manager.cc:194] Registered new tserver with Master: 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
I20260810 01:57:52.703944 25885 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:56142:
table { table_name: "default.Table_983bf2e604df4fdbac75c5aca292f736" } new_table_name: "default.Table_df285e759e764cdeb38de2aad0d1873d" modify_external_catalogs: true
I20260810 01:57:52.714221 23746 tablet_service.cc:1558] Processing DeleteTablet for tablet 81e090d01d8f43cf8cd38e376b7c18ad with delete_type TABLET_DATA_DELETED (replaced by tablet 9b595b2ec2ff4503a20c1e9e7a7467b7) from {username='slave'} at 127.0.0.1:42578
I20260810 01:57:52.716092 23745 tablet_service.cc:1511] Processing CreateTablet for tablet 9b595b2ec2ff4503a20c1e9e7a7467b7 (DEFAULT_TABLE table=default.table_57697192c9bf44969c2b200e94865a3e [id=ef2e836fffc64aa6a8ddc0425f39e317]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:52.716542 23745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b595b2ec2ff4503a20c1e9e7a7467b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:52.727550 23962 tablet_service.cc:1558] Processing DeleteTablet for tablet 81e090d01d8f43cf8cd38e376b7c18ad with delete_type TABLET_DATA_DELETED (replaced by tablet 9b595b2ec2ff4503a20c1e9e7a7467b7) from {username='slave'} at 127.0.0.1:43274
I20260810 01:57:52.730998 26358 tablet_replica.cc:333] T 81e090d01d8f43cf8cd38e376b7c18ad P 48e33bc1e8e94d4887bb64214d0b6c36: stopping tablet replica
I20260810 01:57:52.731237 26358 raft_consensus.cc:2243] T 81e090d01d8f43cf8cd38e376b7c18ad P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 01:57:52.731480 26358 raft_consensus.cc:2272] T 81e090d01d8f43cf8cd38e376b7c18ad P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:57:52.731408 23962 tablet_service.cc:1511] Processing CreateTablet for tablet 9b595b2ec2ff4503a20c1e9e7a7467b7 (DEFAULT_TABLE table=default.table_57697192c9bf44969c2b200e94865a3e [id=ef2e836fffc64aa6a8ddc0425f39e317]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:52.731844 23962 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b595b2ec2ff4503a20c1e9e7a7467b7. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7f6b5681548c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260810 01:57:52.738152 26358 ts_tablet_manager.cc:1916] T 81e090d01d8f43cf8cd38e376b7c18ad P 48e33bc1e8e94d4887bb64214d0b6c36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:57:52.738250 24186 tablet_service.cc:1558] Processing DeleteTablet for tablet 81e090d01d8f43cf8cd38e376b7c18ad with delete_type TABLET_DATA_DELETED (replaced by tablet 9b595b2ec2ff4503a20c1e9e7a7467b7) from {username='slave'} at 127.0.0.1:46954
I20260810 01:57:52.738610 24185 tablet_service.cc:1511] Processing CreateTablet for tablet 9b595b2ec2ff4503a20c1e9e7a7467b7 (DEFAULT_TABLE table=default.table_57697192c9bf44969c2b200e94865a3e [id=ef2e836fffc64aa6a8ddc0425f39e317]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:57:52.739113 24185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b595b2ec2ff4503a20c1e9e7a7467b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:57:52.742677 26356 tablet_bootstrap.cc:492] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 50d44445234b4931aaa28141cdaa67ab: Bootstrap starting.
I20260810 01:57:52.744765 26356 tablet_bootstrap.cc:654] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 50d44445234b4931aaa28141cdaa67ab: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:52.747426 26354 tablet_replica.cc:333] T 81e090d01d8f43cf8cd38e376b7c18ad P 50d44445234b4931aaa28141cdaa67ab: stopping tablet replica
I20260810 01:57:52.747640 26354 raft_consensus.cc:2243] T 81e090d01d8f43cf8cd38e376b7c18ad P 50d44445234b4931aaa28141cdaa67ab [term 1 LEADER]: Raft consensus shutting down.
I20260810 01:57:52.748165 26354 raft_consensus.cc:2272] T 81e090d01d8f43cf8cd38e376b7c18ad P 50d44445234b4931aaa28141cdaa67ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:57:52.748425 26361 tablet_bootstrap.cc:492] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36: Bootstrap starting.
I20260810 01:57:52.748776 26354 ts_tablet_manager.cc:1916] T 81e090d01d8f43cf8cd38e376b7c18ad P 50d44445234b4931aaa28141cdaa67ab: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7f6b56815309 std::_Function_handler<>::_M_invoke() at ??:0
@ 0xaaf86d std::function<>::operator()() at ??:?
I20260810 01:57:52.756167 26358 ts_tablet_manager.cc:1929] T 81e090d01d8f43cf8cd38e376b7c18ad P 48e33bc1e8e94d4887bb64214d0b6c36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 01:57:52.756328 26358 log.cc:1199] T 81e090d01d8f43cf8cd38e376b7c18ad P 48e33bc1e8e94d4887bb64214d0b6c36: Deleting WAL directory at /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-1/wal/wals/81e090d01d8f43cf8cd38e376b7c18ad
I20260810 01:57:52.759441 26361 tablet_bootstrap.cc:654] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:52.760099 26364 tablet_bootstrap.cc:492] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 498d352dbea9405199a121e84a4ab926: Bootstrap starting.
I20260810 01:57:52.760380 26356 tablet_bootstrap.cc:492] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 50d44445234b4931aaa28141cdaa67ab: No bootstrap required, opened a new log
I20260810 01:57:52.760522 26356 ts_tablet_manager.cc:1403] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 50d44445234b4931aaa28141cdaa67ab: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.000s
I20260810 01:57:52.761415 26362 tablet_replica.cc:333] T 81e090d01d8f43cf8cd38e376b7c18ad P 498d352dbea9405199a121e84a4ab926: stopping tablet replica
I20260810 01:57:52.761387 26356 raft_consensus.cc:359] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:52.761595 26356 raft_consensus.cc:385] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:52.761615 26362 raft_consensus.cc:2243] T 81e090d01d8f43cf8cd38e376b7c18ad P 498d352dbea9405199a121e84a4ab926 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 01:57:52.761688 26356 raft_consensus.cc:740] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50d44445234b4931aaa28141cdaa67ab, State: Initialized, Role: FOLLOWER
I20260810 01:57:52.761948 26362 raft_consensus.cc:2272] T 81e090d01d8f43cf8cd38e376b7c18ad P 498d352dbea9405199a121e84a4ab926 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:57:52.762532 26362 ts_tablet_manager.cc:1916] T 81e090d01d8f43cf8cd38e376b7c18ad P 498d352dbea9405199a121e84a4ab926: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:57:52.762792 26358 ts_tablet_manager.cc:1950] T 81e090d01d8f43cf8cd38e376b7c18ad P 48e33bc1e8e94d4887bb64214d0b6c36: Deleting consensus metadata
I20260810 01:57:52.763236 26364 tablet_bootstrap.cc:654] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 498d352dbea9405199a121e84a4ab926: Neither blocks nor log segments found. Creating new log.
I20260810 01:57:52.764140 25864 catalog_manager.cc:5050] TS 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): tablet 81e090d01d8f43cf8cd38e376b7c18ad (table default.table_57697192c9bf44969c2b200e94865a3e [id=ef2e836fffc64aa6a8ddc0425f39e317]) successfully deleted
I20260810 01:57:52.770910 26362 ts_tablet_manager.cc:1929] T 81e090d01d8f43cf8cd38e376b7c18ad P 498d352dbea9405199a121e84a4ab926: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 01:57:52.771883 26361 tablet_bootstrap.cc:492] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36: No bootstrap required, opened a new log
I20260810 01:57:52.772061 26361 ts_tablet_manager.cc:1403] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36: Time spent bootstrapping tablet: real 0.024s user 0.000s sys 0.005s
I20260810 01:57:52.772006 26356 consensus_queue.cc:260] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 50d44445234b4931aaa28141cdaa67ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:52.773119 26356 ts_tablet_manager.cc:1434] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 50d44445234b4931aaa28141cdaa67ab: Time spent starting tablet: real 0.012s user 0.002s sys 0.000s
I20260810 01:57:52.773125 26361 raft_consensus.cc:359] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:52.773370 26361 raft_consensus.cc:385] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:52.773466 26361 raft_consensus.cc:740] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48e33bc1e8e94d4887bb64214d0b6c36, State: Initialized, Role: FOLLOWER
I20260810 01:57:52.773823 26361 consensus_queue.cc:260] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:52.776360 26354 ts_tablet_manager.cc:1929] T 81e090d01d8f43cf8cd38e376b7c18ad P 50d44445234b4931aaa28141cdaa67ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 01:57:52.776477 26354 log.cc:1199] T 81e090d01d8f43cf8cd38e376b7c18ad P 50d44445234b4931aaa28141cdaa67ab: Deleting WAL directory at /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-0/wal/wals/81e090d01d8f43cf8cd38e376b7c18ad
I20260810 01:57:52.776923 26354 ts_tablet_manager.cc:1950] T 81e090d01d8f43cf8cd38e376b7c18ad P 50d44445234b4931aaa28141cdaa67ab: Deleting consensus metadata
I20260810 01:57:52.778970 25864 catalog_manager.cc:5050] TS 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): tablet 81e090d01d8f43cf8cd38e376b7c18ad (table default.table_57697192c9bf44969c2b200e94865a3e [id=ef2e836fffc64aa6a8ddc0425f39e317]) successfully deleted
I20260810 01:57:52.783135 26362 log.cc:1199] T 81e090d01d8f43cf8cd38e376b7c18ad P 498d352dbea9405199a121e84a4ab926: Deleting WAL directory at /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-2/wal/wals/81e090d01d8f43cf8cd38e376b7c18ad
I20260810 01:57:52.783819 26362 ts_tablet_manager.cc:1950] T 81e090d01d8f43cf8cd38e376b7c18ad P 498d352dbea9405199a121e84a4ab926: Deleting consensus metadata
I20260810 01:57:52.785302 25867 catalog_manager.cc:5050] TS 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): tablet 81e090d01d8f43cf8cd38e376b7c18ad (table default.table_57697192c9bf44969c2b200e94865a3e [id=ef2e836fffc64aa6a8ddc0425f39e317]) successfully deleted
I20260810 01:57:52.793090 24069 heartbeater.cc:344] Connected to a master server at 127.22.69.61:41945
I20260810 01:57:52.796197 26229 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" instance_seqno: 1786327048775850) as {username='slave'} at 127.22.69.2:43827; Asking this server to re-register.
I20260810 01:57:52.797372 24069 heartbeater.cc:461] Registering TS with master...
I20260810 01:57:52.799708 26229 ts_manager.cc:194] Registered new tserver with Master: 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
I20260810 01:57:52.802232 26361 ts_tablet_manager.cc:1434] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36: Time spent starting tablet: real 0.030s user 0.000s sys 0.003s
@ 0x7f6b4286ddd9 kudu::Thread::SuperviseThread() at ??:0
I20260810 01:57:52.818455 23846 heartbeater.cc:344] Connected to a master server at 127.22.69.61:41945
I20260810 01:57:52.821038 26229 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" instance_seqno: 1786327048176732) as {username='slave'} at 127.22.69.1:44423; Asking this server to re-register.
I20260810 01:57:52.821977 23846 heartbeater.cc:461] Registering TS with master...
I20260810 01:57:52.823740 26229 ts_manager.cc:194] Registered new tserver with Master: 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491)
@ 0x7f6b45f3d6db start_thread at ??:0
@ 0x7f6b3e9f461f clone at ??:0
I20260810 01:57:52.835427 26364 tablet_bootstrap.cc:492] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 498d352dbea9405199a121e84a4ab926: No bootstrap required, opened a new log
I20260810 01:57:52.835608 26364 ts_tablet_manager.cc:1403] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 498d352dbea9405199a121e84a4ab926: Time spent bootstrapping tablet: real 0.076s user 0.020s sys 0.039s
I20260810 01:57:52.836532 26364 raft_consensus.cc:359] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:52.836732 26364 raft_consensus.cc:385] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:57:52.836817 26364 raft_consensus.cc:740] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 498d352dbea9405199a121e84a4ab926, State: Initialized, Role: FOLLOWER
I20260810 01:57:52.837055 26364 consensus_queue.cc:260] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 498d352dbea9405199a121e84a4ab926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:52.837559 24300 heartbeater.cc:499] Master 127.22.69.62:35311 was elected leader, sending a full tablet report...
I20260810 01:57:52.839560 26364 ts_tablet_manager.cc:1434] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 498d352dbea9405199a121e84a4ab926: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260810 01:57:52.905818 26181 raft_consensus.cc:493] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 01:57:52.906036 26181 raft_consensus.cc:515] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:52.906788 26181 leader_election.cc:290] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 01:57:52.908288 23892 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
W20260810 01:57:52.908526 23889 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
I20260810 01:57:52.908675 23889 leader_election.cc:304] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:57:52.909119 26181 raft_consensus.cc:2749] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 01:57:52.999419 26146 raft_consensus.cc:493] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:57:52.999625 26146 raft_consensus.cc:515] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:53.000430 26146 leader_election.cc:290] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
I20260810 01:57:53.000962 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b595b2ec2ff4503a20c1e9e7a7467b7" candidate_uuid: "498d352dbea9405199a121e84a4ab926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50d44445234b4931aaa28141cdaa67ab" is_pre_election: true
I20260810 01:57:53.000982 23997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b595b2ec2ff4503a20c1e9e7a7467b7" candidate_uuid: "498d352dbea9405199a121e84a4ab926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" is_pre_election: true
I20260810 01:57:53.001186 23997 raft_consensus.cc:2468] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 498d352dbea9405199a121e84a4ab926 in term 0.
I20260810 01:57:53.001214 23776 raft_consensus.cc:2468] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 498d352dbea9405199a121e84a4ab926 in term 0.
I20260810 01:57:53.001732 24109 leader_election.cc:304] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab; no voters:
I20260810 01:57:53.002056 26146 raft_consensus.cc:2804] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 01:57:53.002180 26146 raft_consensus.cc:493] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 01:57:53.002276 26146 raft_consensus.cc:3060] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:53.004546 26146 raft_consensus.cc:515] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 498d352dbea9405199a121e84a4ab926 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:53.005141 26146 leader_election.cc:290] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 election: Requested vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
I20260810 01:57:53.005671 23997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b595b2ec2ff4503a20c1e9e7a7467b7" candidate_uuid: "498d352dbea9405199a121e84a4ab926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48e33bc1e8e94d4887bb64214d0b6c36"
I20260810 01:57:53.005867 23997 raft_consensus.cc:3060] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:53.005975 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b595b2ec2ff4503a20c1e9e7a7467b7" candidate_uuid: "498d352dbea9405199a121e84a4ab926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50d44445234b4931aaa28141cdaa67ab"
I20260810 01:57:53.006191 23776 raft_consensus.cc:3060] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 50d44445234b4931aaa28141cdaa67ab [term 0 FOLLOWER]: Advancing to term 1
I20260810 01:57:53.009027 23997 raft_consensus.cc:2468] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 498d352dbea9405199a121e84a4ab926 in term 1.
I20260810 01:57:53.009466 24109 leader_election.cc:304] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36, 498d352dbea9405199a121e84a4ab926; no voters:
I20260810 01:57:53.009742 23776 raft_consensus.cc:2468] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 50d44445234b4931aaa28141cdaa67ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 498d352dbea9405199a121e84a4ab926 in term 1.
I20260810 01:57:53.009826 26146 raft_consensus.cc:2804] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 498d352dbea9405199a121e84a4ab926 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 01:57:53.011358 26146 raft_consensus.cc:697] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 498d352dbea9405199a121e84a4ab926 [term 1 LEADER]: Becoming Leader. State: Replica: 498d352dbea9405199a121e84a4ab926, State: Running, Role: LEADER
I20260810 01:57:53.011706 26146 consensus_queue.cc:237] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 498d352dbea9405199a121e84a4ab926 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
W20260810 01:57:53.024313 24118 connection.cc:573] client connection to 127.22.69.62:35311 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 01:57:53.026046 26165 negotiation.cc:336] Failed RPC negotiation. Trace:
0810 01:57:53.025233 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.22.69.62:35311 (local address 127.22.69.3:51233)
0810 01:57:53.025529 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0810 01:57:53.025549 (+ 20us) client_negotiation.cc:175] Beginning negotiation
0810 01:57:53.025667 (+ 118us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0810 01:57:53.025825 (+ 158us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":162}
W20260810 01:57:53.026211 24321 connection.cc:573] client connection to 127.22.69.62:35311 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 01:57:53.026921 24321 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.22.69.62:35311)
W20260810 01:57:53.027889 24321 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.22.69.62:35311)
W20260810 01:57:53.028327 24321 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.22.69.62:35311)
W20260810 01:57:53.028661 24321 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.22.69.62:35311)
I20260810 01:57:53.201814 24222 raft_consensus.cc:1217] T f88158dec2a449c2a2011b8fc042f0ac P 498d352dbea9405199a121e84a4ab926 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 0.0->[1.1-1.1] Dedup: 1.1->[]
I20260810 01:57:53.214018 26412 raft_consensus.cc:493] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 919313c0dee540a690f46dbc8208d20b)
I20260810 01:57:53.214185 26412 raft_consensus.cc:515] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:53.216192 26412 leader_election.cc:290] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:57:53.217008 26201 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111)
W20260810 01:57:53.218290 26201 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311): Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111)
W20260810 01:57:53.218523 26197 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:57:53.218649 26197 leader_election.cc:304] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1422349180c446db6c1768b3025253c; no voters: 757bb3f336084510845ed58df264c721, 919313c0dee540a690f46dbc8208d20b
I20260810 01:57:53.218923 26412 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260810 01:57:53.496598 23892 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 6: this message will repeat every 5th retry.
I20260810 01:57:53.746311 26420 raft_consensus.cc:493] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 919313c0dee540a690f46dbc8208d20b)
I20260810 01:57:53.746517 26420 raft_consensus.cc:515] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:53.747315 26420 leader_election.cc:290] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:57:53.749275 26201 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311): Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111)
W20260810 01:57:53.749428 26197 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:57:53.749571 26197 leader_election.cc:304] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1422349180c446db6c1768b3025253c; no voters: 757bb3f336084510845ed58df264c721, 919313c0dee540a690f46dbc8208d20b
I20260810 01:57:53.749850 26420 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260810 01:57:53.917618 23889 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Couldn't send request to peer 50d44445234b4931aaa28141cdaa67ab. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 6: this message will repeat every 5th retry.
I20260810 01:57:54.104314 26146 consensus_queue.cc:1048] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 498d352dbea9405199a121e84a4ab926 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:57:54.111392 26435 consensus_queue.cc:1048] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 498d352dbea9405199a121e84a4ab926 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 01:57:54.805534 26460 raft_consensus.cc:493] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 919313c0dee540a690f46dbc8208d20b)
I20260810 01:57:54.805727 26460 raft_consensus.cc:515] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:54.806480 26460 leader_election.cc:290] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:57:54.808950 26201 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311): Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111)
W20260810 01:57:54.809580 26197 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:57:54.809679 26197 leader_election.cc:304] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1422349180c446db6c1768b3025253c; no voters: 757bb3f336084510845ed58df264c721, 919313c0dee540a690f46dbc8208d20b
I20260810 01:57:54.809998 26460 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260810 01:57:54.960064 26467 raft_consensus.cc:493] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:57:54.960276 26467 raft_consensus.cc:515] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:54.960922 26467 leader_election.cc:290] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:57:54.961923 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
W20260810 01:57:54.962174 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
I20260810 01:57:54.962308 24109 leader_election.cc:304] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:57:54.962585 26467 raft_consensus.cc:2749] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:57:55.026908 26449 raft_consensus.cc:493] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 01:57:55.027132 26449 raft_consensus.cc:515] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:55.027697 26449 leader_election.cc:290] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 01:57:55.028553 23889 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
W20260810 01:57:55.028752 23892 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
I20260810 01:57:55.028928 23892 leader_election.cc:304] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:57:55.029196 26449 raft_consensus.cc:2749] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 01:57:55.725796 22804 external_mini_cluster.cc:1372] Running /tmp/dist-test-task_qhVVJ/build/asan/bin/kudu
/tmp/dist-test-task_qhVVJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.69.62:35311
--webserver_interface=127.22.69.62
--webserver_port=40455
--builtin_ntp_servers=127.22.69.20:43209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917
--hive_metastore_uris=thrift://127.0.0.1:37513
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260810 01:57:55.741567 26491 raft_consensus.cc:493] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 919313c0dee540a690f46dbc8208d20b)
I20260810 01:57:55.741788 26491 raft_consensus.cc:515] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:55.742498 26491 leader_election.cc:290] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:57:55.744935 26201 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311): Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111)
W20260810 01:57:55.745127 26197 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:57:55.745237 26197 leader_election.cc:304] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1422349180c446db6c1768b3025253c; no voters: 757bb3f336084510845ed58df264c721, 919313c0dee540a690f46dbc8208d20b
I20260810 01:57:55.745529 26491 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260810 01:57:56.077574 26489 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:57:56.078006 26489 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:57:56.078155 26489 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260810 01:57:56.086843 26489 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:57:56.086962 26489 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:57:56.087091 26489 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:57:56.087177 26489 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:57:56.097524 26489 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.69.20:43209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:37513
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.69.62:35311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.69.62
--webserver_port=40455
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 10 Aug 2026 01:44:04 UTC on e7f111948823
build id 13266
ASAN enabled
I20260810 01:57:56.099182 26489 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:57:56.100989 26489 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260810 01:57:56.109591 26509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:57:56.109516 26510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:57:56.111212 26514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:57:56.111533 26489 server_base.cc:1061] running on GCE node
I20260810 01:57:56.112277 26489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:57:56.113674 26489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:57:56.114928 26489 hybrid_clock.cc:648] HybridClock initialized: now 1786327076114877 us; error 62 us; skew 500 ppm
I20260810 01:57:56.115337 26489 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 01:57:56.118075 26489 webserver.cc:533] Webserver started at http://127.22.69.62:40455/ using document root <none> and password file <none>
I20260810 01:57:56.119079 26489 fs_manager.cc:362] Metadata directory not provided
I20260810 01:57:56.119217 26489 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:57:56.125202 26489 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 01:57:56.130471 26533 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:57:56.131806 26489 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 01:57:56.132009 26489 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/data,/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/wal
uuid: "919313c0dee540a690f46dbc8208d20b"
format_stamp: "Formatted at 2026-08-10 01:57:26 on dist-test-slave-n326"
I20260810 01:57:56.132694 26489 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:57:56.169461 26489 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260810 01:57:56.170379 26489 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:57:56.170754 26489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:57:56.192318 26489 rpc_server.cc:307] RPC server started. Bound to: 127.22.69.62:35311
I20260810 01:57:56.192322 26612 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.22.69.62:35311 every 8 connection(s)
I20260810 01:57:56.193948 26489 server_base.cc:1193] Dumped server information to /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/data/info.pb
I20260810 01:57:56.198411 22804 external_mini_cluster.cc:1434] Started /tmp/dist-test-task_qhVVJ/build/asan/bin/kudu as pid 26489
I20260810 01:57:56.202900 26613 sys_catalog.cc:263] Verifying existing consensus state
I20260810 01:57:56.209286 26613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b: Bootstrap starting.
I20260810 01:57:56.225342 26613 log.cc:826] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b: Log is configured to *not* fsync() on all Append() calls
I20260810 01:57:56.263731 26613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b: Bootstrap replayed 1/1 log segments. Stats: ops{read=49 overwritten=0 applied=49 ignored=0} inserts{seen=32 ignored=0} mutations{seen=35 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 01:57:56.264325 26613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b: Bootstrap complete.
I20260810 01:57:56.269754 26613 raft_consensus.cc:359] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 8 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:56.270510 26613 raft_consensus.cc:740] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 8 FOLLOWER]: Becoming Follower/Learner. State: Replica: 919313c0dee540a690f46dbc8208d20b, State: Initialized, Role: FOLLOWER
I20260810 01:57:56.271263 26613 consensus_queue.cc:260] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 49, Last appended: 8.49, Last appended by leader: 49, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:56.272101 26619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 8 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:56.272358 26619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:57:56.273278 26613 sys_catalog.cc:565] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:57:56.294718 26644 catalog_manager.cc:1295] Loaded cluster ID: dfa48c56b63b4d0cbf3d6124921e8a17
I20260810 01:57:56.294831 26644 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b: loading cluster ID for follower catalog manager: success
I20260810 01:57:56.299633 26644 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b: acquiring CA information for follower catalog manager: success
I20260810 01:57:56.303345 26644 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 01:57:56.321414 26489 master_runner.cc:187] Error getting master registration for 127.22.69.60:36917: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111),
I20260810 01:57:56.329674 26489 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 01:57:56.330653 24066 heartbeater.cc:344] Connected to a master server at 127.22.69.62:35311
I20260810 01:57:56.334189 26559 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" instance_seqno: 1786327048775850) as {username='slave'} at 127.22.69.2:34521; Asking this server to re-register.
I20260810 01:57:56.334995 24066 heartbeater.cc:461] Registering TS with master...
I20260810 01:57:56.344409 26557 ts_manager.cc:194] Registered new tserver with Master: 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
I20260810 01:57:56.396191 24300 heartbeater.cc:344] Connected to a master server at 127.22.69.62:35311
I20260810 01:57:56.396386 24300 heartbeater.cc:499] Master 127.22.69.62:35311 was elected leader, sending a full tablet report...
I20260810 01:57:56.398871 26557 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "498d352dbea9405199a121e84a4ab926" instance_seqno: 1786327049319804) as {username='slave'} at 127.22.69.3:48227; Asking this server to re-register.
I20260810 01:57:56.399724 24300 heartbeater.cc:461] Registering TS with master...
I20260810 01:57:56.401376 26557 ts_manager.cc:194] Registered new tserver with Master: 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
I20260810 01:57:56.598081 26467 raft_consensus.cc:493] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:57:56.598299 26467 raft_consensus.cc:515] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:56.598977 26467 leader_election.cc:290] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:57:56.599892 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
W20260810 01:57:56.600179 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
I20260810 01:57:56.600303 24109 leader_election.cc:304] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:57:56.600594 26467 raft_consensus.cc:2749] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:57:56.627408 23842 heartbeater.cc:344] Connected to a master server at 127.22.69.62:35311
I20260810 01:57:56.629263 26557 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" instance_seqno: 1786327048176732) as {username='slave'} at 127.22.69.1:55653; Asking this server to re-register.
I20260810 01:57:56.629969 23842 heartbeater.cc:461] Registering TS with master...
I20260810 01:57:56.631417 26557 ts_manager.cc:194] Registered new tserver with Master: 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491)
I20260810 01:57:56.701004 26619 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:57:56.701241 26619 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:56.703025 26619 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
I20260810 01:57:56.703648 26243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "919313c0dee540a690f46dbc8208d20b" candidate_term: 9 candidate_status { last_received { term: 8 index: 49 } } ignore_live_leader: false dest_uuid: "b1422349180c446db6c1768b3025253c" is_pre_election: true
I20260810 01:57:56.703907 26243 raft_consensus.cc:2468] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 919313c0dee540a690f46dbc8208d20b in term 8.
W20260810 01:57:56.703907 26536 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:57:56.704780 26536 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b, b1422349180c446db6c1768b3025253c; no voters:
I20260810 01:57:56.705135 26619 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260810 01:57:56.705260 26619 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 01:57:56.705402 26619 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 8 FOLLOWER]: Advancing to term 9
W20260810 01:57:56.705693 26536 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:57:56.709357 26619 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:56.709921 26619 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 9 election: Requested vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
I20260810 01:57:56.710381 26243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "919313c0dee540a690f46dbc8208d20b" candidate_term: 9 candidate_status { last_received { term: 8 index: 49 } } ignore_live_leader: false dest_uuid: "b1422349180c446db6c1768b3025253c"
I20260810 01:57:56.710536 26243 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 8 FOLLOWER]: Advancing to term 9
W20260810 01:57:56.711465 26536 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:57:56.712695 26243 raft_consensus.cc:2468] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 919313c0dee540a690f46dbc8208d20b in term 9.
I20260810 01:57:56.713140 26536 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b, b1422349180c446db6c1768b3025253c; no voters: 757bb3f336084510845ed58df264c721
I20260810 01:57:56.713443 26619 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 9 FOLLOWER]: Leader election won for term 9
I20260810 01:57:56.713855 26619 raft_consensus.cc:697] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 9 LEADER]: Becoming Leader. State: Replica: 919313c0dee540a690f46dbc8208d20b, State: Running, Role: LEADER
I20260810 01:57:56.714295 26619 consensus_queue.cc:237] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 49, Committed index: 49, Last appended: 8.49, Last appended by leader: 49, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:56.717033 26666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 919313c0dee540a690f46dbc8208d20b. Latest consensus state: current_term: 9 leader_uuid: "919313c0dee540a690f46dbc8208d20b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:56.717281 26666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: This master's current role is: LEADER
I20260810 01:57:56.718009 26669 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260810 01:57:56.720885 26669 catalog_manager.cc:679] Loaded metadata for table default.table_72b6f93a124042e4a090dc194134c75a [id=017756566b72415aa33842f8df2b3e91]
I20260810 01:57:56.721350 26669 catalog_manager.cc:679] Loaded metadata for table default.table_73d521a2c5504815ad6d7cc3dd0bf818 [id=2f1832807601444a88d61b8396a67db9]
I20260810 01:57:56.721732 26669 catalog_manager.cc:679] Loaded metadata for table default.table_c8ad906be0f74210912a57adead21255 [id=4da3c920ea6a48588c9f0c91da00fb69]
I20260810 01:57:56.722110 26669 catalog_manager.cc:679] Loaded metadata for table default.table_983bf2e604df4fdbac75c5aca292f736 [id=552771dde220474a9be9abeefd3a0472]
I20260810 01:57:56.722448 26669 catalog_manager.cc:679] Loaded metadata for table default.table_22726fd1a11f409f9f5287ab16d8e551 [id=5edf9cb336414ae3bd41e9f5ca41fcca]
I20260810 01:57:56.722805 26669 catalog_manager.cc:679] Loaded metadata for table default.table_0ba2046f969346f68f1bde7cf6c9f273 [id=630b03060b7646118927a18573c0920c]
I20260810 01:57:56.723201 26669 catalog_manager.cc:679] Loaded metadata for table default.table_510ae1b5fcc74db788ffb64925cad250 [id=7d240d10089c47eaa014d59c4ac7f0b4]
I20260810 01:57:56.723564 26669 catalog_manager.cc:679] Loaded metadata for table default.table_379ea52611ef49d591a022addb7afc0c [id=87ba140e9080402aa494198d6f212f05]
I20260810 01:57:56.723905 26669 catalog_manager.cc:679] Loaded metadata for table default.table_57697192c9bf44969c2b200e94865a3e [id=ef2e836fffc64aa6a8ddc0425f39e317]
I20260810 01:57:56.726230 26669 tablet_loader.cc:96] loaded metadata for tablet 1c3db8429f9f4cf5b29d3d1a6cc0651c (table default.table_379ea52611ef49d591a022addb7afc0c [id=87ba140e9080402aa494198d6f212f05])
I20260810 01:57:56.726649 26669 tablet_loader.cc:96] loaded metadata for tablet 2230d919953e49208659450a024c6a62 (table default.table_0ba2046f969346f68f1bde7cf6c9f273 [id=630b03060b7646118927a18573c0920c])
I20260810 01:57:56.726940 26669 tablet_loader.cc:96] loaded metadata for tablet 4514cc930ba44e0f92a438f8ae0ba6fc (table default.table_510ae1b5fcc74db788ffb64925cad250 [id=7d240d10089c47eaa014d59c4ac7f0b4])
I20260810 01:57:56.727409 26669 tablet_loader.cc:96] loaded metadata for tablet 5974ae70170544f797ff749280b3be0c (table default.table_983bf2e604df4fdbac75c5aca292f736 [id=552771dde220474a9be9abeefd3a0472])
I20260810 01:57:56.727670 26669 tablet_loader.cc:96] loaded metadata for tablet 5c49a207b37e40579d34002c3fb57afb (table default.table_c8ad906be0f74210912a57adead21255 [id=4da3c920ea6a48588c9f0c91da00fb69])
I20260810 01:57:56.727900 26669 tablet_loader.cc:96] loaded metadata for tablet 6ca335f1e5964f8490a49ba4c4fb4c72 (table default.table_72b6f93a124042e4a090dc194134c75a [id=017756566b72415aa33842f8df2b3e91])
I20260810 01:57:56.728457 26669 tablet_loader.cc:96] loaded metadata for tablet 9b595b2ec2ff4503a20c1e9e7a7467b7 (table default.table_57697192c9bf44969c2b200e94865a3e [id=ef2e836fffc64aa6a8ddc0425f39e317])
I20260810 01:57:56.729200 26669 tablet_loader.cc:96] loaded metadata for tablet cad2404759604bfea7c273e1aa40539a (table default.table_73d521a2c5504815ad6d7cc3dd0bf818 [id=2f1832807601444a88d61b8396a67db9])
I20260810 01:57:56.729640 26669 tablet_loader.cc:96] loaded metadata for tablet f88158dec2a449c2a2011b8fc042f0ac (table default.table_22726fd1a11f409f9f5287ab16d8e551 [id=5edf9cb336414ae3bd41e9f5ca41fcca])
I20260810 01:57:56.729854 26669 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260810 01:57:56.730733 26669 catalog_manager.cc:1295] Loaded cluster ID: dfa48c56b63b4d0cbf3d6124921e8a17
I20260810 01:57:56.730806 26669 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260810 01:57:56.732192 26669 catalog_manager.cc:1540] Loading token signing keys...
I20260810 01:57:56.733013 26669 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b: Loaded TSK: 0
I20260810 01:57:56.733832 26669 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260810 01:57:56.734658 26669 catalog_manager.cc:1563] Loading latest processed Hive Metastore notification log event ID...
I20260810 01:57:56.735539 26669 catalog_manager.cc:5941] Last processed Hive Metastore notification event ID: -1
I20260810 01:57:56.771668 26559 master_service.cc:960] ReplaceTablet: received request to replace tablet 09ed68395e2d44a5b51a22abe2203406 from {username='slave'} at 127.0.0.1:56902
I20260810 01:57:56.771889 26556 master_service.cc:960] ReplaceTablet: received request to replace tablet 2230d919953e49208659450a024c6a62 from {username='slave'} at 127.0.0.1:56902
I20260810 01:57:56.772711 26554 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56902:
name: "default.Table_e3c77b251fdf471d8916f06da94be990"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:56.773183 26548 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56902:
name: "default.Table_70288f170151490fb6ba63a3bf29f01e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:56.771661 26557 master_service.cc:960] ReplaceTablet: received request to replace tablet 89d9556cc526444299c786de606bb5b1 from {username='slave'} at 127.0.0.1:56902
I20260810 01:57:56.774049 26555 master_service.cc:960] ReplaceTablet: received request to replace tablet 89d9556cc526444299c786de606bb5b1 from {username='slave'} at 127.0.0.1:56902
I20260810 01:57:56.774689 26559 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:56902:
table { table_name: "default.Table_c8ad906be0f74210912a57adead21255" } new_table_name: "default.Table_e2cb378ea960473c833b1d18fbe32f91" modify_external_catalogs: true
I20260810 01:57:56.775087 26552 master_service.cc:960] ReplaceTablet: received request to replace tablet e6ef3afe40a847dbb08792578d52401d from {username='slave'} at 127.0.0.1:56902
I20260810 01:57:56.775234 26557 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:56902:
table { table_name: "default.Table_72b6f93a124042e4a090dc194134c75a" } new_table_name: "default.Table_b8900cd2c2574605ade34fc7ccc46f94" modify_external_catalogs: true
I20260810 01:57:56.773355 26546 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:56902:
table { table_name: "default.Table_73d521a2c5504815ad6d7cc3dd0bf818" } new_table_name: "default.Table_f5851b3fa264482388e4e50f34d4e37c" modify_external_catalogs: true
I20260810 01:57:56.775656 26549 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56902:
table { table_name: "default.Table_22726fd1a11f409f9f5287ab16d8e551" } modify_external_catalogs: true
I20260810 01:57:56.775178 26550 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56902:
name: "default.Table_5a4f0e1bcc16485ab9a7148d13d21a68"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:56.776707 26549 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56902:
table { table_name: "default.Table_22726fd1a11f409f9f5287ab16d8e551" } modify_external_catalogs: true
F20260810 01:57:56.784662 26643 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 10 received events: NotificationEvent(eventId=1, eventTime=1786327049, eventType=CREATE_TABLE, dbName=default, tableName=table_4fb63c687c11493ca7c0b76735f137e9, message={"server":"","servicePrincipal":"","db":"default","table":"table_4fb63c687c11493ca7c0b76735f137e9","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_4fb63c687c11493ca7c0b76735f137e9\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1786327049},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/warehouse/table_4fb63c687c11493ca7c0b76735f137e9\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_4fb63c687c11493ca7c0b76735f137e9\",\"kudu.table_id\":\"cb924727e6014a2a9dd271f962df3807\",\"transient_lastDdlTime\":\"1786327049\",\"comment\":\"\",\"kudu.master_addresses\":\"127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"dfa48c56b63b4d0cbf3d6124921e8a17\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1786327049,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1786327049, eventType=CREATE_TABLE, dbName=default, tableName=table_c8ad906be0f74210912a57adead21255, message={"server":"","servicePrincipal":"","db":"default","table":"table_c8ad906be0f74210912a57adead21255","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_c8ad906be0f74210912a57adead21255\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1786327049},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/warehouse/table_c8ad906be0f74210912a57adead21255\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_c8ad906be0f74210912a57adead21255\",\"kudu.table_id\":\"4da3c920ea6a48588c9f0c91da00fb69\",\"transient_lastDdlTime\":\"1786327049\",\"comment\":\"\",\"kudu.master_addresses\":\"127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"dfa48c56b63b4d0cbf3d6124921e8a17\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1786327049,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
I20260810 01:57:56.780440 26555 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56902:
name: "default.Table_bb2d4e224bd14be09d60bce7977d4c19"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:57:56.783972 26552 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56902:
name: "default.Table_1bd23f5807da48238cbddc7597b8ab50"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
@ 0x7f5e1530ed5d google::LogMessage::Fail() at ??:0
@ 0x7f5e15312b23 google::LogMessage::SendToLog() at ??:0
@ 0x7f5e1530e75c google::LogMessage::Flush() at ??:0
I20260810 01:57:56.792151 26243 raft_consensus.cc:1275] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 9 FOLLOWER]: Refusing update from remote peer 919313c0dee540a690f46dbc8208d20b: Log matching property violated. Preceding OpId in replica: term: 8 index: 49. Preceding OpId from leader: term: 9 index: 51. (index mismatch)
W20260810 01:57:56.792757 26536 consensus_peers.cc:597] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b -> Peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Couldn't send request to peer 757bb3f336084510845ed58df264c721. Status: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
@ 0x7f5e1530e9a1 google::LogMessage::~LogMessage() at ??:0
I20260810 01:57:56.796424 26666 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 50, Last known committed idx: 49, Time since last communication: 0.000s
@ 0x7f5e2bfd7f64 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7f5e2bfd6811 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f5e2bfde48c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7f5e2bfde309 std::_Function_handler<>::_M_invoke() at ??:0
@ 0xaaf86d std::function<>::operator()() at ??:?
@ 0x7f5e18036dd9 kudu::Thread::SuperviseThread() at ??:0
I20260810 01:57:56.823307 26678 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 919313c0dee540a690f46dbc8208d20b. Latest consensus state: current_term: 9 leader_uuid: "919313c0dee540a690f46dbc8208d20b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:56.823611 26678 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: This master's current role is: FOLLOWER
@ 0x7f5e1b7066db start_thread at ??:0
@ 0x7f5e141bd61f clone at ??:0
I20260810 01:57:56.826710 26678 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 9 leader_uuid: "919313c0dee540a690f46dbc8208d20b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:57:56.826905 26678 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: This master's current role is: FOLLOWER
W20260810 01:57:57.004347 24118 connection.cc:573] client connection to 127.22.69.62:35311 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 01:57:57.004350 26201 connection.cc:573] server connection from 127.0.0.1:60522 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 01:57:57.004698 23892 connection.cc:573] client connection to 127.22.69.62:35311 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 01:57:57.004832 24321 connection.cc:573] client connection to 127.22.69.62:35311 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 01:57:57.005524 24321 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.22.69.62:35311)
W20260810 01:57:57.006074 24321 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.22.69.62:35311)
W20260810 01:57:57.006292 24321 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.22.69.62:35311)
W20260810 01:57:57.006711 24321 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.22.69.62:35311)
I20260810 01:57:57.326218 26678 raft_consensus.cc:493] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 919313c0dee540a690f46dbc8208d20b)
I20260810 01:57:57.326396 26678 raft_consensus.cc:515] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:57.327225 26678 leader_election.cc:290] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:57:57.329794 26201 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311): Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111)
W20260810 01:57:57.329993 26197 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:57:57.330121 26197 leader_election.cc:304] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1422349180c446db6c1768b3025253c; no voters: 757bb3f336084510845ed58df264c721, 919313c0dee540a690f46dbc8208d20b
I20260810 01:57:57.330369 26678 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260810 01:57:57.609668 26158 master_runner.cc:187] Error getting master registration for 127.22.69.62:35311: Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111),
I20260810 01:57:57.609869 26158 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 01:57:57.897789 26721 raft_consensus.cc:493] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 919313c0dee540a690f46dbc8208d20b)
I20260810 01:57:57.897975 26721 raft_consensus.cc:515] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:57.898541 26721 leader_election.cc:290] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:57:57.900354 26201 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311): Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111)
W20260810 01:57:57.900535 26197 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:57:57.900650 26197 leader_election.cc:304] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1422349180c446db6c1768b3025253c; no voters: 757bb3f336084510845ed58df264c721, 919313c0dee540a690f46dbc8208d20b
I20260810 01:57:57.900883 26721 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260810 01:57:58.533798 26733 raft_consensus.cc:493] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 919313c0dee540a690f46dbc8208d20b)
I20260810 01:57:58.533988 26733 raft_consensus.cc:515] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:58.534768 26733 leader_election.cc:290] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:57:58.535831 26197 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111) [suppressed 11 similar messages]
W20260810 01:57:58.537595 26197 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
W20260810 01:57:58.537801 26201 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311): Network error: Client connection negotiation failed: client connection to 127.22.69.62:35311: connect: Connection refused (error 111)
I20260810 01:57:58.537959 26201 leader_election.cc:304] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1422349180c446db6c1768b3025253c; no voters: 757bb3f336084510845ed58df264c721, 919313c0dee540a690f46dbc8208d20b
I20260810 01:57:58.538255 26733 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260810 01:57:58.750200 26730 raft_consensus.cc:493] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:57:58.750438 26730 raft_consensus.cc:515] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:57:58.751350 26730 leader_election.cc:290] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:57:58.752290 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
W20260810 01:57:58.752571 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
I20260810 01:57:58.752679 24109 leader_election.cc:304] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:57:58.752967 26730 raft_consensus.cc:2749] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:57:59.078497 26726 raft_consensus.cc:493] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 01:57:59.078701 26726 raft_consensus.cc:515] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:57:59.079658 26726 leader_election.cc:290] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 01:57:59.080488 23889 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
W20260810 01:57:59.080689 23892 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
I20260810 01:57:59.080828 23892 leader_election.cc:304] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:57:59.081067 26726 raft_consensus.cc:2749] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 01:57:59.464193 22804 external_mini_cluster.cc:1372] Running /tmp/dist-test-task_qhVVJ/build/asan/bin/kudu
/tmp/dist-test-task_qhVVJ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.69.62:35311
--webserver_interface=127.22.69.62
--webserver_port=40455
--builtin_ntp_servers=127.22.69.20:43209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917
--hive_metastore_uris=thrift://127.0.0.1:37513
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260810 01:57:59.816643 26745 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 01:57:59.817040 26745 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 01:57:59.817195 26745 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260810 01:57:59.826502 26745 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 01:57:59.826623 26745 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260810 01:57:59.826762 26745 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 01:57:59.826853 26745 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260810 01:57:59.837826 26745 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.69.20:43209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:37513
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.69.62:35311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.69.62
--webserver_port=40455
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 10 Aug 2026 01:44:04 UTC on e7f111948823
build id 13266
ASAN enabled
I20260810 01:57:59.839790 26745 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 01:57:59.841929 26745 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260810 01:57:59.850150 26759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:57:59.850186 26756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 01:57:59.851154 26761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 01:57:59.851444 26745 server_base.cc:1061] running on GCE node
I20260810 01:57:59.852165 26745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 01:57:59.853477 26745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 01:57:59.854732 26745 hybrid_clock.cc:648] HybridClock initialized: now 1786327079854672 us; error 65 us; skew 500 ppm
I20260810 01:57:59.855157 26745 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 01:57:59.857241 26745 webserver.cc:533] Webserver started at http://127.22.69.62:40455/ using document root <none> and password file <none>
I20260810 01:57:59.857959 26745 fs_manager.cc:362] Metadata directory not provided
I20260810 01:57:59.858110 26745 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 01:57:59.863813 26745 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 01:57:59.866704 26769 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 01:57:59.868042 26745 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 01:57:59.868219 26745 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/data,/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/wal
uuid: "919313c0dee540a690f46dbc8208d20b"
format_stamp: "Formatted at 2026-08-10 01:57:26 on dist-test-slave-n326"
I20260810 01:57:59.868901 26745 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 01:57:59.895581 26745 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260810 01:57:59.896374 26745 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 01:57:59.896687 26745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 01:57:59.919672 26745 rpc_server.cc:307] RPC server started. Bound to: 127.22.69.62:35311
I20260810 01:57:59.919708 26853 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.22.69.62:35311 every 8 connection(s)
I20260810 01:57:59.921371 26745 server_base.cc:1193] Dumped server information to /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/master-0/data/info.pb
I20260810 01:57:59.926693 22804 external_mini_cluster.cc:1434] Started /tmp/dist-test-task_qhVVJ/build/asan/bin/kudu as pid 26745
I20260810 01:57:59.929092 26854 sys_catalog.cc:263] Verifying existing consensus state
I20260810 01:57:59.932622 26854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b: Bootstrap starting.
I20260810 01:57:59.948911 26854 log.cc:826] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b: Log is configured to *not* fsync() on all Append() calls
I20260810 01:57:59.964135 26864 raft_consensus.cc:493] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 919313c0dee540a690f46dbc8208d20b)
I20260810 01:57:59.964363 26864 raft_consensus.cc:515] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:57:59.965173 26864 leader_election.cc:290] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:57:59.968566 26197 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:57:59.990353 26854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b: Bootstrap replayed 1/1 log segments. Stats: ops{read=51 overwritten=0 applied=50 ignored=0} inserts{seen=32 ignored=0} mutations{seen=35 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260810 01:57:59.991240 26854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b: Bootstrap complete.
I20260810 01:57:59.996865 26854 raft_consensus.cc:359] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 9 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:58:00.000010 26854 raft_consensus.cc:740] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 9 FOLLOWER]: Becoming Follower/Learner. State: Replica: 919313c0dee540a690f46dbc8208d20b, State: Initialized, Role: FOLLOWER
I20260810 01:58:00.000837 26854 consensus_queue.cc:260] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 50, Last appended: 9.51, Last appended by leader: 51, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:58:00.001729 26871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 9 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:58:00.002050 26871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 01:58:00.002825 26854 sys_catalog.cc:565] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 01:58:00.002827 26812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b1422349180c446db6c1768b3025253c" candidate_term: 10 candidate_status { last_received { term: 9 index: 51 } } ignore_live_leader: false dest_uuid: "919313c0dee540a690f46dbc8208d20b" is_pre_election: true
I20260810 01:58:00.003490 26812 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1422349180c446db6c1768b3025253c in term 9.
I20260810 01:58:00.004261 26201 leader_election.cc:304] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b, b1422349180c446db6c1768b3025253c; no voters: 757bb3f336084510845ed58df264c721
I20260810 01:58:00.004673 26864 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260810 01:58:00.004827 26864 raft_consensus.cc:493] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 9 FOLLOWER]: Starting leader election (detected failure of leader 919313c0dee540a690f46dbc8208d20b)
I20260810 01:58:00.004940 26864 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 9 FOLLOWER]: Advancing to term 10
I20260810 01:58:00.007668 26864 raft_consensus.cc:515] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:58:00.008421 26864 leader_election.cc:290] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 10 election: Requested vote from peers 919313c0dee540a690f46dbc8208d20b (127.22.69.62:35311), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
I20260810 01:58:00.008809 26812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b1422349180c446db6c1768b3025253c" candidate_term: 10 candidate_status { last_received { term: 9 index: 51 } } ignore_live_leader: false dest_uuid: "919313c0dee540a690f46dbc8208d20b"
I20260810 01:58:00.008992 26812 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 9 FOLLOWER]: Advancing to term 10
W20260810 01:58:00.010293 26197 leader_election.cc:336] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:58:00.014183 26812 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1422349180c446db6c1768b3025253c in term 10.
I20260810 01:58:00.014652 26201 leader_election.cc:304] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b, b1422349180c446db6c1768b3025253c; no voters: 757bb3f336084510845ed58df264c721
I20260810 01:58:00.014953 26864 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 10 FOLLOWER]: Leader election won for term 10
I20260810 01:58:00.022322 24300 heartbeater.cc:344] Connected to a master server at 127.22.69.62:35311
I20260810 01:58:00.026618 26797 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "498d352dbea9405199a121e84a4ab926" instance_seqno: 1786327049319804) as {username='slave'} at 127.22.69.3:36189; Asking this server to re-register.
I20260810 01:58:00.027839 24300 heartbeater.cc:461] Registering TS with master...
I20260810 01:58:00.029866 26798 ts_manager.cc:194] Registered new tserver with Master: 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
I20260810 01:58:00.033746 26864 raft_consensus.cc:697] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [term 10 LEADER]: Becoming Leader. State: Replica: b1422349180c446db6c1768b3025253c, State: Running, Role: LEADER
I20260810 01:58:00.034830 26864 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 51, Committed index: 51, Last appended: 9.51, Last appended by leader: 51, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:58:00.043298 26897 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1422349180c446db6c1768b3025253c. Latest consensus state: current_term: 10 leader_uuid: "b1422349180c446db6c1768b3025253c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:58:00.043696 26897 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: This master's current role is: LEADER
I20260810 01:58:00.043828 26896 catalog_manager.cc:1295] Loaded cluster ID: dfa48c56b63b4d0cbf3d6124921e8a17
I20260810 01:58:00.043917 26896 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b: loading cluster ID for follower catalog manager: success
I20260810 01:58:00.048410 26896 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b: acquiring CA information for follower catalog manager: success
I20260810 01:58:00.051888 26896 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 01:58:00.053100 26908 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260810 01:58:00.056529 26908 catalog_manager.cc:679] Loaded metadata for table default.table_72b6f93a124042e4a090dc194134c75a [id=017756566b72415aa33842f8df2b3e91]
I20260810 01:58:00.057219 26908 catalog_manager.cc:679] Loaded metadata for table default.table_73d521a2c5504815ad6d7cc3dd0bf818 [id=2f1832807601444a88d61b8396a67db9]
I20260810 01:58:00.057812 26908 catalog_manager.cc:679] Loaded metadata for table default.table_c8ad906be0f74210912a57adead21255 [id=4da3c920ea6a48588c9f0c91da00fb69]
I20260810 01:58:00.058318 26908 catalog_manager.cc:679] Loaded metadata for table default.table_983bf2e604df4fdbac75c5aca292f736 [id=552771dde220474a9be9abeefd3a0472]
I20260810 01:58:00.058847 26908 catalog_manager.cc:679] Loaded metadata for table default.table_22726fd1a11f409f9f5287ab16d8e551 [id=5edf9cb336414ae3bd41e9f5ca41fcca]
I20260810 01:58:00.059366 26908 catalog_manager.cc:679] Loaded metadata for table default.table_0ba2046f969346f68f1bde7cf6c9f273 [id=630b03060b7646118927a18573c0920c]
I20260810 01:58:00.059849 26908 catalog_manager.cc:679] Loaded metadata for table default.table_510ae1b5fcc74db788ffb64925cad250 [id=7d240d10089c47eaa014d59c4ac7f0b4]
I20260810 01:58:00.060273 26908 catalog_manager.cc:679] Loaded metadata for table default.table_379ea52611ef49d591a022addb7afc0c [id=87ba140e9080402aa494198d6f212f05]
I20260810 01:58:00.060796 26908 catalog_manager.cc:679] Loaded metadata for table default.table_57697192c9bf44969c2b200e94865a3e [id=ef2e836fffc64aa6a8ddc0425f39e317]
I20260810 01:58:00.064119 26908 tablet_loader.cc:96] loaded metadata for tablet 1c3db8429f9f4cf5b29d3d1a6cc0651c (table default.table_379ea52611ef49d591a022addb7afc0c [id=87ba140e9080402aa494198d6f212f05])
I20260810 01:58:00.064795 26908 tablet_loader.cc:96] loaded metadata for tablet 4514cc930ba44e0f92a438f8ae0ba6fc (table default.table_510ae1b5fcc74db788ffb64925cad250 [id=7d240d10089c47eaa014d59c4ac7f0b4])
I20260810 01:58:00.065442 26908 tablet_loader.cc:96] loaded metadata for tablet 5974ae70170544f797ff749280b3be0c (table default.table_983bf2e604df4fdbac75c5aca292f736 [id=552771dde220474a9be9abeefd3a0472])
I20260810 01:58:00.065887 26908 tablet_loader.cc:96] loaded metadata for tablet 5c49a207b37e40579d34002c3fb57afb (table default.table_c8ad906be0f74210912a57adead21255 [id=4da3c920ea6a48588c9f0c91da00fb69])
I20260810 01:58:00.066316 26908 tablet_loader.cc:96] loaded metadata for tablet 6ca335f1e5964f8490a49ba4c4fb4c72 (table default.table_72b6f93a124042e4a090dc194134c75a [id=017756566b72415aa33842f8df2b3e91])
I20260810 01:58:00.067243 26908 tablet_loader.cc:96] loaded metadata for tablet 9b595b2ec2ff4503a20c1e9e7a7467b7 (table default.table_57697192c9bf44969c2b200e94865a3e [id=ef2e836fffc64aa6a8ddc0425f39e317])
I20260810 01:58:00.068396 26908 tablet_loader.cc:96] loaded metadata for tablet cad2404759604bfea7c273e1aa40539a (table default.table_73d521a2c5504815ad6d7cc3dd0bf818 [id=2f1832807601444a88d61b8396a67db9])
I20260810 01:58:00.068732 26908 tablet_loader.cc:96] loaded metadata for tablet d6cdef730e234b67843af88af2d1a6c2 (table default.table_0ba2046f969346f68f1bde7cf6c9f273 [id=630b03060b7646118927a18573c0920c])
I20260810 01:58:00.069396 26908 tablet_loader.cc:96] loaded metadata for tablet f88158dec2a449c2a2011b8fc042f0ac (table default.table_22726fd1a11f409f9f5287ab16d8e551 [id=5edf9cb336414ae3bd41e9f5ca41fcca])
I20260810 01:58:00.069722 26908 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260810 01:58:00.071131 26908 catalog_manager.cc:1295] Loaded cluster ID: dfa48c56b63b4d0cbf3d6124921e8a17
I20260810 01:58:00.071244 26908 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260810 01:58:00.072955 26908 catalog_manager.cc:1540] Loading token signing keys...
I20260810 01:58:00.074215 26908 catalog_manager.cc:6103] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c: Loaded TSK: 0
I20260810 01:58:00.075428 26908 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260810 01:58:00.076699 26908 catalog_manager.cc:1563] Loading latest processed Hive Metastore notification log event ID...
I20260810 01:58:00.077934 26908 catalog_manager.cc:5941] Last processed Hive Metastore notification event ID: -1
I20260810 01:58:00.072540 26745 master_runner.cc:187] Error getting master registration for 127.22.69.60:36917: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111),
I20260810 01:58:00.078202 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 01:58:00.127192 26226 master_service.cc:960] ReplaceTablet: received request to replace tablet f88158dec2a449c2a2011b8fc042f0ac from {username='slave'} at 127.0.0.1:60498
I20260810 01:58:00.128293 26220 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60498:
table { table_name: "default.Table_379ea52611ef49d591a022addb7afc0c" } new_table_name: "default.Table_09a7f5150ded428991a5d464657e85ae" modify_external_catalogs: true
I20260810 01:58:00.128185 26221 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60498:
name: "default.Table_73822ba0bac443d19cc7c66f6f547bda"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:58:00.128940 26229 master_service.cc:960] ReplaceTablet: received request to replace tablet 9b595b2ec2ff4503a20c1e9e7a7467b7 from {username='slave'} at 127.0.0.1:60498
I20260810 01:58:00.129087 26223 master_service.cc:960] ReplaceTablet: received request to replace tablet 16db112abfb34d4a9f2ecbee98c39d8b from {username='slave'} at 127.0.0.1:60498
I20260810 01:58:00.128909 26225 master_service.cc:960] ReplaceTablet: received request to replace tablet 9b595b2ec2ff4503a20c1e9e7a7467b7 from {username='slave'} at 127.0.0.1:60498
I20260810 01:58:00.130614 26212 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60498:
table { table_name: "default.Table_983bf2e604df4fdbac75c5aca292f736" } new_table_name: "default.Table_df285e759e764cdeb38de2aad0d1873d" modify_external_catalogs: true
I20260810 01:58:00.132103 26223 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60498:
table { table_name: "default.Table_0ba2046f969346f68f1bde7cf6c9f273" } modify_external_catalogs: true
I20260810 01:58:00.132424 26223 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60498:
table { table_name: "default.Table_0ba2046f969346f68f1bde7cf6c9f273" } modify_external_catalogs: true
I20260810 01:58:00.133134 26208 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60498:
name: "default.Table_b03802d8f2f14a24b00ab6efbe9392dc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:58:00.133226 26211 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60498:
name: "default.Table_9175c2bd9b0542f2a6b95eae8a882b7a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:58:00.134054 26207 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60498:
name: "default.Table_14c5bada100e48adb5f378e73ee7d00b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:58:00.137985 26812 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Refusing update from remote peer b1422349180c446db6c1768b3025253c: Log matching property violated. Preceding OpId in replica: term: 9 index: 51. Preceding OpId from leader: term: 10 index: 53. (index mismatch)
W20260810 01:58:00.138383 26197 consensus_peers.cc:597] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c -> Peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Couldn't send request to peer 757bb3f336084510845ed58df264c721. Status: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 01:58:00.138993 26897 consensus_queue.cc:1048] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [LEADER]: Connected to new peer: Peer: permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 52, Last known committed idx: 50, Time since last communication: 0.000s
F20260810 01:58:00.144097 26319 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 10 received events: NotificationEvent(eventId=1, eventTime=1786327049, eventType=CREATE_TABLE, dbName=default, tableName=table_4fb63c687c11493ca7c0b76735f137e9, message={"server":"","servicePrincipal":"","db":"default","table":"table_4fb63c687c11493ca7c0b76735f137e9","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_4fb63c687c11493ca7c0b76735f137e9\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1786327049},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/warehouse/table_4fb63c687c11493ca7c0b76735f137e9\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_4fb63c687c11493ca7c0b76735f137e9\",\"kudu.table_id\":\"cb924727e6014a2a9dd271f962df3807\",\"transient_lastDdlTime\":\"1786327049\",\"comment\":\"\",\"kudu.master_addresses\":\"127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"dfa48c56b63b4d0cbf3d6124921e8a17\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1786327049,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1786327049, eventType=CREATE_TABLE, dbName=default, tableName=table_c8ad906be0f74210912a57adead21255, message={"server":"","servicePrincipal":"","db":"default","table":"table_c8ad906be0f74210912a57adead21255","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_c8ad906be0f74210912a57adead21255\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1786327049},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/warehouse/table_c8ad906be0f74210912a57adead21255\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_c8ad906be0f74210912a57adead21255\",\"kudu.table_id\":\"4da3c920ea6a48588c9f0c91da00fb69\",\"transient_lastDdlTime\":\"1786327049\",\"comment\":\"\",\"kudu.master_addresses\":\"127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"dfa48c56b63b4d0cbf3d6124921e8a17\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1786327049,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
@ 0x7f5da7002d5d google::LogMessage::Fail() at ??:0
@ 0x7f5da7006b23 google::LogMessage::SendToLog() at ??:0
I20260810 01:58:00.151170 26871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1422349180c446db6c1768b3025253c. Latest consensus state: current_term: 10 leader_uuid: "b1422349180c446db6c1768b3025253c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:58:00.151420 26871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: This master's current role is: FOLLOWER
@ 0x7f5da700275c google::LogMessage::Flush() at ??:0
I20260810 01:58:00.154228 23842 heartbeater.cc:344] Connected to a master server at 127.22.69.62:35311
@ 0x7f5da70029a1 google::LogMessage::~LogMessage() at ??:0
I20260810 01:58:00.156050 26909 mvcc.cc:204] Tried to move back new op lower bound from 7316795720235962368 to 7316795719828930560. Current Snapshot: MvccSnapshot[applied={T|T < 7316795720235962368}]
I20260810 01:58:00.157598 26870 mvcc.cc:204] Tried to move back new op lower bound from 7316795720243068928 to 7316795719828930560. Current Snapshot: MvccSnapshot[applied={T|T < 7316795720235962368}]
I20260810 01:58:00.169912 26229 catalog_manager.cc:6006] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c: Sending DeleteTablet for 3 replicas of tablet 9b595b2ec2ff4503a20c1e9e7a7467b7
I20260810 01:58:00.170090 26864 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 10 leader_uuid: "b1422349180c446db6c1768b3025253c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:58:00.170403 26864 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c [sys.catalog]: This master's current role is: LEADER
@ 0x7f5dbdccbf64 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260810 01:58:00.171967 26229 catalog_manager.cc:6807] ReplaceTablet: tablet 9b595b2ec2ff4503a20c1e9e7a7467b7 deleted and replaced by tablet 0dc1d64a95b84b26b17a29a67b5e3a8d
I20260810 01:58:00.172307 26229 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60498:
table { table_name: "default.Table_57697192c9bf44969c2b200e94865a3e" } modify_external_catalogs: true
I20260810 01:58:00.173046 26225 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60498:
name: "default.Table_e3c77b251fdf471d8916f06da94be990"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:58:00.172586 26229 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60498:
table { table_name: "default.Table_57697192c9bf44969c2b200e94865a3e" } modify_external_catalogs: true
@ 0x7f5dbdcca811 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260810 01:58:00.180500 26226 catalog_manager.cc:6006] T 00000000000000000000000000000000 P b1422349180c446db6c1768b3025253c: Sending DeleteTablet for 3 replicas of tablet f88158dec2a449c2a2011b8fc042f0ac
I20260810 01:58:00.181090 26226 catalog_manager.cc:6807] ReplaceTablet: tablet f88158dec2a449c2a2011b8fc042f0ac deleted and replaced by tablet fe2481b03e0841a2916c6a53c307bbfb
I20260810 01:58:00.181370 26226 master_service.cc:960] ReplaceTablet: received request to replace tablet 2230d919953e49208659450a024c6a62 from {username='slave'} at 127.0.0.1:60498
I20260810 01:58:00.182056 26226 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60498:
name: "default.Table_5a4f0e1bcc16485ab9a7148d13d21a68"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 01:58:00.188517 26797 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" instance_seqno: 1786327048176732) as {username='slave'} at 127.22.69.1:34507; Asking this server to re-register.
@ 0x7f5dbdcd248c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260810 01:58:00.191879 26871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 10 leader_uuid: "b1422349180c446db6c1768b3025253c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } } }
I20260810 01:58:00.192104 26871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [sys.catalog]: This master's current role is: FOLLOWER
@ 0x7f5dbdcd2309 std::_Function_handler<>::_M_invoke() at ??:0
I20260810 01:58:00.193830 24185 tablet_service.cc:1558] Processing DeleteTablet for tablet 9b595b2ec2ff4503a20c1e9e7a7467b7 with delete_type TABLET_DATA_DELETED (replaced by tablet 0dc1d64a95b84b26b17a29a67b5e3a8d) from {username='slave'} at 127.0.0.1:50484
I20260810 01:58:00.194173 24186 tablet_service.cc:1558] Processing DeleteTablet for tablet f88158dec2a449c2a2011b8fc042f0ac with delete_type TABLET_DATA_DELETED (replaced by tablet fe2481b03e0841a2916c6a53c307bbfb) from {username='slave'} at 127.0.0.1:50484
@ 0xaaf86d std::function<>::operator()() at ??:?
@ 0x7f5da9d2add9 kudu::Thread::SuperviseThread() at ??:0
I20260810 01:58:00.207262 26933 tablet_replica.cc:333] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 498d352dbea9405199a121e84a4ab926: stopping tablet replica
I20260810 01:58:00.207573 26933 raft_consensus.cc:2243] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 498d352dbea9405199a121e84a4ab926 [term 1 LEADER]: Raft consensus shutting down.
I20260810 01:58:00.208308 24185 tablet_service.cc:1511] Processing CreateTablet for tablet 0dc1d64a95b84b26b17a29a67b5e3a8d (DEFAULT_TABLE table=default.table_57697192c9bf44969c2b200e94865a3e [id=ef2e836fffc64aa6a8ddc0425f39e317]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:58:00.208987 24185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0dc1d64a95b84b26b17a29a67b5e3a8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:58:00.209198 24186 tablet_service.cc:1511] Processing CreateTablet for tablet d6cdef730e234b67843af88af2d1a6c2 (DEFAULT_TABLE table=default.table_0ba2046f969346f68f1bde7cf6c9f273 [id=630b03060b7646118927a18573c0920c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 01:58:00.209619 24186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6cdef730e234b67843af88af2d1a6c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 01:58:00.211634 26933 raft_consensus.cc:2272] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 498d352dbea9405199a121e84a4ab926 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:58:00.212260 26933 ts_tablet_manager.cc:1916] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 498d352dbea9405199a121e84a4ab926: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7f5dad3fa6db start_thread at ??:0
@ 0x7f5da5eb161f clone at ??:0
I20260810 01:58:00.220021 26935 tablet_bootstrap.cc:492] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926: Bootstrap starting.
I20260810 01:58:00.222299 26935 tablet_bootstrap.cc:654] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926: Neither blocks nor log segments found. Creating new log.
I20260810 01:58:00.224784 26933 ts_tablet_manager.cc:1929] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 498d352dbea9405199a121e84a4ab926: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 01:58:00.224920 26933 log.cc:1199] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 498d352dbea9405199a121e84a4ab926: Deleting WAL directory at /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-2/wal/wals/9b595b2ec2ff4503a20c1e9e7a7467b7
I20260810 01:58:00.225488 26933 ts_tablet_manager.cc:1950] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 498d352dbea9405199a121e84a4ab926: Deleting consensus metadata
I20260810 01:58:00.226718 26933 tablet_replica.cc:333] T f88158dec2a449c2a2011b8fc042f0ac P 498d352dbea9405199a121e84a4ab926: stopping tablet replica
I20260810 01:58:00.226954 26933 raft_consensus.cc:2243] T f88158dec2a449c2a2011b8fc042f0ac P 498d352dbea9405199a121e84a4ab926 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 01:58:00.227679 23842 heartbeater.cc:461] Registering TS with master...
I20260810 01:58:00.228258 26933 raft_consensus.cc:2272] T f88158dec2a449c2a2011b8fc042f0ac P 498d352dbea9405199a121e84a4ab926 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 01:58:00.228966 26933 ts_tablet_manager.cc:1916] T f88158dec2a449c2a2011b8fc042f0ac P 498d352dbea9405199a121e84a4ab926: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 01:58:00.229475 26797 ts_manager.cc:194] Registered new tserver with Master: 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491)
I20260810 01:58:00.235824 26935 tablet_bootstrap.cc:492] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926: No bootstrap required, opened a new log
I20260810 01:58:00.235962 26935 ts_tablet_manager.cc:1403] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.000s
I20260810 01:58:00.236752 26935 raft_consensus.cc:359] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:00.236935 26935 raft_consensus.cc:385] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:58:00.237023 26935 raft_consensus.cc:740] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 498d352dbea9405199a121e84a4ab926, State: Initialized, Role: FOLLOWER
I20260810 01:58:00.237280 26935 consensus_queue.cc:260] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:00.239538 26935 ts_tablet_manager.cc:1434] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 01:58:00.239809 26935 tablet_bootstrap.cc:492] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926: Bootstrap starting.
I20260810 01:58:00.241513 26935 tablet_bootstrap.cc:654] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926: Neither blocks nor log segments found. Creating new log.
I20260810 01:58:00.245281 26933 ts_tablet_manager.cc:1929] T f88158dec2a449c2a2011b8fc042f0ac P 498d352dbea9405199a121e84a4ab926: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 01:58:00.245431 26933 log.cc:1199] T f88158dec2a449c2a2011b8fc042f0ac P 498d352dbea9405199a121e84a4ab926: Deleting WAL directory at /tmp/dist-test-task_qhVVJ/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786327028618185-22804-0/minicluster-data/ts-2/wal/wals/f88158dec2a449c2a2011b8fc042f0ac
I20260810 01:58:00.246676 26935 tablet_bootstrap.cc:492] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926: No bootstrap required, opened a new log
I20260810 01:58:00.246829 26935 ts_tablet_manager.cc:1403] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260810 01:58:00.247614 26935 raft_consensus.cc:359] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:00.247813 26935 raft_consensus.cc:385] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 01:58:00.247918 26935 raft_consensus.cc:740] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 498d352dbea9405199a121e84a4ab926, State: Initialized, Role: FOLLOWER
I20260810 01:58:00.248203 26935 consensus_queue.cc:260] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:00.249541 26935 ts_tablet_manager.cc:1434] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260810 01:58:00.249884 26933 ts_tablet_manager.cc:1950] T f88158dec2a449c2a2011b8fc042f0ac P 498d352dbea9405199a121e84a4ab926: Deleting consensus metadata
I20260810 01:58:00.362754 26730 raft_consensus.cc:493] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:58:00.363198 26730 raft_consensus.cc:515] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:00.364110 26730 leader_election.cc:290] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:58:00.366628 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
W20260810 01:58:00.366878 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
I20260810 01:58:00.366981 24109 leader_election.cc:304] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:00.367554 26730 raft_consensus.cc:2749] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:58:00.385233 26730 raft_consensus.cc:493] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:58:00.385418 26730 raft_consensus.cc:515] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:00.386195 26730 leader_election.cc:290] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:58:00.388224 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
W20260810 01:58:00.388476 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
I20260810 01:58:00.388623 24109 leader_election.cc:304] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:00.388916 26730 raft_consensus.cc:2749] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:58:00.412762 24066 heartbeater.cc:344] Connected to a master server at 127.22.69.62:35311
I20260810 01:58:00.415120 26797 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" instance_seqno: 1786327048775850) as {username='slave'} at 127.22.69.2:51611; Asking this server to re-register.
I20260810 01:58:00.415918 24066 heartbeater.cc:461] Registering TS with master...
I20260810 01:58:00.418013 26797 ts_manager.cc:194] Registered new tserver with Master: 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:58:00.420454 24302 tablet.cc:2406] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 01:58:00.431411 24111 connection.cc:573] server connection from 127.0.0.1:50484 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 01:58:00.431962 26931 negotiation.cc:336] Failed RPC negotiation. Trace:
0810 01:58:00.175754 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:56358 (local address 127.22.69.2:46271)
0810 01:58:00.202439 (+ 26685us) server_negotiation.cc:207] Beginning negotiation
0810 01:58:00.202447 (+ 8us) server_negotiation.cc:400] Waiting for connection header
0810 01:58:00.202477 (+ 30us) server_negotiation.cc:408] Connection header received
0810 01:58:00.202554 (+ 77us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0810 01:58:00.202561 (+ 7us) server_negotiation.cc:462] Received NEGOTIATE request from client
0810 01:58:00.202643 (+ 82us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0810 01:58:00.205916 (+ 3273us) server_negotiation.cc:227] Negotiated authn=CERTIFICATE
0810 01:58:00.206190 (+ 274us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0810 01:58:00.211212 (+ 5022us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0810 01:58:00.431543 (+220331us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56358: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":26551,"thread_start_us":31317,"threads_started":1}
W20260810 01:58:00.432139 23890 connection.cc:573] client connection to 127.22.69.61:41945 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 01:58:00.432638 24111 connection.cc:573] client connection to 127.22.69.61:41945 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 01:58:00.432731 26928 negotiation.cc:336] Failed RPC negotiation. Trace:
0810 01:58:00.172519 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:57840 (local address 127.22.69.1:45491)
0810 01:58:00.227472 (+ 54953us) server_negotiation.cc:207] Beginning negotiation
0810 01:58:00.227478 (+ 6us) server_negotiation.cc:400] Waiting for connection header
0810 01:58:00.227496 (+ 18us) server_negotiation.cc:408] Connection header received
0810 01:58:00.227556 (+ 60us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0810 01:58:00.227561 (+ 5us) server_negotiation.cc:462] Received NEGOTIATE request from client
0810 01:58:00.227645 (+ 84us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0810 01:58:00.227763 (+ 118us) server_negotiation.cc:227] Negotiated authn=CERTIFICATE
0810 01:58:00.431599 (+203836us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57840: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":54837,"thread_start_us":54696,"threads_started":1}
W20260810 01:58:00.433935 24317 connection.cc:573] client connection to 127.22.69.61:41945 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 01:58:00.434669 24317 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.22.69.61:41945)
W20260810 01:58:00.435382 24317 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.22.69.61:41945)
W20260810 01:58:00.435642 24317 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.22.69.61:41945)
W20260810 01:58:00.435972 24317 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.22.69.61:41945)
W20260810 01:58:00.551271 23684 consensus_peers.cc:597] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f88158dec2a449c2a2011b8fc042f0ac. This is attempt 1: this message will repeat every 5th retry.
I20260810 01:58:00.674281 23846 heartbeater.cc:499] Master 127.22.69.61:41945 was elected leader, sending a full tablet report...
I20260810 01:58:00.710683 26957 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 01:58:00.710889 26957 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:58:00.712581 26957 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:58:00.713304 26771 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 01:58:00.714708 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 01:58:00.714951 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:58:00.715073 26771 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 01:58:00.715438 26957 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260810 01:58:00.845471 26730 raft_consensus.cc:493] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:58:00.845712 26730 raft_consensus.cc:515] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:00.846515 26730 leader_election.cc:290] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:58:00.847725 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
W20260810 01:58:00.848070 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
I20260810 01:58:00.848177 24109 leader_election.cc:304] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:00.848470 26730 raft_consensus.cc:2749] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:58:01.406847 26963 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 01:58:01.407115 26963 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:58:01.407794 26963 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:58:01.409950 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 01:58:01.410175 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:58:01.410303 26771 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 01:58:01.410591 26963 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260810 01:58:01.617780 26969 raft_consensus.cc:493] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 01:58:01.618021 26969 raft_consensus.cc:515] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:01.618752 26969 leader_election.cc:290] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 01:58:01.619904 23889 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
W20260810 01:58:01.620069 23892 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
I20260810 01:58:01.620190 23892 leader_election.cc:304] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:01.620431 26969 raft_consensus.cc:2749] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 01:58:02.193192 22804 master-stress-test.cc:579] Killing test thread 1/25
I20260810 01:58:02.413631 26991 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 01:58:02.413837 26991 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:58:02.414487 26991 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:58:02.417121 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 01:58:02.417368 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:58:02.417457 26771 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 01:58:02.417701 26991 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260810 01:58:02.723340 26998 raft_consensus.cc:493] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 50d44445234b4931aaa28141cdaa67ab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 01:58:02.723572 26998 raft_consensus.cc:515] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 50d44445234b4931aaa28141cdaa67ab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:02.724393 26998 leader_election.cc:290] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 50d44445234b4931aaa28141cdaa67ab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
I20260810 01:58:02.725008 23997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b595b2ec2ff4503a20c1e9e7a7467b7" candidate_uuid: "50d44445234b4931aaa28141cdaa67ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" is_pre_election: true
W20260810 01:58:02.725443 23684 leader_election.cc:343] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 50d44445234b4931aaa28141cdaa67ab [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 9b595b2ec2ff4503a20c1e9e7a7467b7
I20260810 01:58:02.725716 23679 leader_election.cc:304] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 50d44445234b4931aaa28141cdaa67ab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50d44445234b4931aaa28141cdaa67ab; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 498d352dbea9405199a121e84a4ab926
I20260810 01:58:02.725996 26998 raft_consensus.cc:2749] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 50d44445234b4931aaa28141cdaa67ab [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 01:58:03.114343 27004 raft_consensus.cc:493] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 01:58:03.114545 27004 raft_consensus.cc:515] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:03.115355 27004 leader_election.cc:290] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
I20260810 01:58:03.115968 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b595b2ec2ff4503a20c1e9e7a7467b7" candidate_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "50d44445234b4931aaa28141cdaa67ab" is_pre_election: true
I20260810 01:58:03.116219 23776 raft_consensus.cc:2468] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 50d44445234b4931aaa28141cdaa67ab [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48e33bc1e8e94d4887bb64214d0b6c36 in term 1.
W20260810 01:58:03.116518 23892 leader_election.cc:343] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 9b595b2ec2ff4503a20c1e9e7a7467b7
I20260810 01:58:03.116835 23889 leader_election.cc:304] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab; no voters: 498d352dbea9405199a121e84a4ab926
I20260810 01:58:03.117161 27004 raft_consensus.cc:2804] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 01:58:03.117281 27004 raft_consensus.cc:493] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 01:58:03.117404 27004 raft_consensus.cc:3060] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Advancing to term 2
I20260810 01:58:03.119632 27004 raft_consensus.cc:515] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:03.120186 27004 leader_election.cc:290] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 election: Requested vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
I20260810 01:58:03.120604 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b595b2ec2ff4503a20c1e9e7a7467b7" candidate_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "50d44445234b4931aaa28141cdaa67ab"
I20260810 01:58:03.120795 23776 raft_consensus.cc:3060] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 50d44445234b4931aaa28141cdaa67ab [term 1 FOLLOWER]: Advancing to term 2
W20260810 01:58:03.121053 23892 leader_election.cc:343] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 9b595b2ec2ff4503a20c1e9e7a7467b7
I20260810 01:58:03.122886 23776 raft_consensus.cc:2468] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 50d44445234b4931aaa28141cdaa67ab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48e33bc1e8e94d4887bb64214d0b6c36 in term 2.
I20260810 01:58:03.123358 23889 leader_election.cc:304] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab; no voters: 498d352dbea9405199a121e84a4ab926
I20260810 01:58:03.123656 27004 raft_consensus.cc:2804] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 01:58:03.123847 27004 raft_consensus.cc:697] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 2 LEADER]: Becoming Leader. State: Replica: 48e33bc1e8e94d4887bb64214d0b6c36, State: Running, Role: LEADER
I20260810 01:58:03.124195 27004 consensus_queue.cc:237] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:03.393198 27015 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 01:58:03.393468 27015 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:58:03.394290 27015 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:58:03.396580 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 01:58:03.396754 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:58:03.396849 26771 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 01:58:03.397123 27015 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260810 01:58:03.756096 23889 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Couldn't send request to peer 50d44445234b4931aaa28141cdaa67ab. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 11: this message will repeat every 5th retry.
W20260810 01:58:03.896073 23892 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 11: this message will repeat every 5th retry.
I20260810 01:58:03.979667 23776 raft_consensus.cc:1275] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 50d44445234b4931aaa28141cdaa67ab [term 2 FOLLOWER]: Refusing update from remote peer 48e33bc1e8e94d4887bb64214d0b6c36: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260810 01:58:03.980351 27034 consensus_queue.cc:1048] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260810 01:58:03.986644 23892 consensus_peers.cc:597] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9b595b2ec2ff4503a20c1e9e7a7467b7. This is attempt 1: this message will repeat every 5th retry.
I20260810 01:58:04.243335 26730 raft_consensus.cc:493] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:58:04.243528 26730 raft_consensus.cc:515] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:04.244248 26730 leader_election.cc:290] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:58:04.245270 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
W20260810 01:58:04.245642 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
I20260810 01:58:04.245798 24109 leader_election.cc:304] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:04.246088 26730 raft_consensus.cc:2749] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:58:04.357231 26730 raft_consensus.cc:493] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:58:04.357393 26730 raft_consensus.cc:515] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:04.357966 26730 leader_election.cc:290] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:58:04.359019 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
W20260810 01:58:04.359323 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
I20260810 01:58:04.359447 24109 leader_election.cc:304] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:04.359715 26730 raft_consensus.cc:2749] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:58:04.449673 26730 raft_consensus.cc:493] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:58:04.449857 26730 raft_consensus.cc:515] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:04.450702 26730 leader_election.cc:290] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:58:04.451776 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
W20260810 01:58:04.452015 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
I20260810 01:58:04.452152 24109 leader_election.cc:304] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:04.452430 26730 raft_consensus.cc:2749] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:58:05.065014 27071 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 01:58:05.065258 27071 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:58:05.065984 27071 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:58:05.071378 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 01:58:05.071650 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:58:05.071831 26771 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 01:58:05.072311 27071 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260810 01:58:05.080487 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 01:58:05.080662 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 01:58:06.550486 27090 raft_consensus.cc:493] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:58:06.550712 27090 raft_consensus.cc:515] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:06.551469 27090 leader_election.cc:290] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:58:06.552707 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
W20260810 01:58:06.552948 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
I20260810 01:58:06.553071 24109 leader_election.cc:304] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:06.553363 27090 raft_consensus.cc:2749] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:58:07.775444 27125 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 01:58:07.775646 27125 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:58:07.776404 27125 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:58:07.777683 26771 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111) [suppressed 9 similar messages]
W20260810 01:58:07.779242 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 01:58:07.779498 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:58:07.779609 26771 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 01:58:07.779841 27125 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260810 01:58:08.032423 27141 raft_consensus.cc:493] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 01:58:08.032663 27141 raft_consensus.cc:515] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:58:08.033635 27141 leader_election.cc:290] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 01:58:08.034462 23889 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
W20260810 01:58:08.034658 23892 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
I20260810 01:58:08.034847 23892 leader_election.cc:304] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:08.035173 27141 raft_consensus.cc:2749] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 01:58:08.722235 27155 raft_consensus.cc:493] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:58:08.722407 27155 raft_consensus.cc:515] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:08.723104 27155 leader_election.cc:290] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:58:08.724085 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
W20260810 01:58:08.724335 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
I20260810 01:58:08.724486 24109 leader_election.cc:304] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:08.724757 27155 raft_consensus.cc:2749] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:58:09.400578 27155 raft_consensus.cc:493] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:58:09.400843 27155 raft_consensus.cc:515] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:09.401556 27155 leader_election.cc:290] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:58:09.402765 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
W20260810 01:58:09.403029 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
I20260810 01:58:09.403228 24109 leader_election.cc:304] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:09.403617 27155 raft_consensus.cc:2749] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:58:10.083927 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 01:58:10.084167 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 01:58:10.217505 27187 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 01:58:10.217767 27187 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:58:10.218562 27187 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:58:10.220502 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 01:58:10.220702 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:58:10.220795 26771 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 01:58:10.221011 27187 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260810 01:58:10.764801 23684 consensus_peers.cc:597] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f88158dec2a449c2a2011b8fc042f0ac. This is attempt 6: this message will repeat every 5th retry.
I20260810 01:58:10.943207 27155 raft_consensus.cc:493] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:58:10.943396 27155 raft_consensus.cc:515] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:10.944025 27155 leader_election.cc:290] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:58:10.944941 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
W20260810 01:58:10.945142 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
I20260810 01:58:10.945235 24109 leader_election.cc:304] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:10.945502 27155 raft_consensus.cc:2749] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:58:11.020448 27155 raft_consensus.cc:493] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:58:11.020766 27155 raft_consensus.cc:515] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:11.021508 27155 leader_election.cc:290] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:58:11.022555 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
W20260810 01:58:11.022828 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
I20260810 01:58:11.022923 24109 leader_election.cc:304] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:11.023258 27155 raft_consensus.cc:2749] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:58:11.051429 27182 raft_consensus.cc:493] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 01:58:11.051618 27182 raft_consensus.cc:515] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:58:11.052323 27182 leader_election.cc:290] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 01:58:11.053238 23889 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
W20260810 01:58:11.053375 23892 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
I20260810 01:58:11.053558 23892 leader_election.cc:304] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:11.053871 27182 raft_consensus.cc:2749] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 01:58:11.568370 27155 raft_consensus.cc:493] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:58:11.568621 27155 raft_consensus.cc:515] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:11.569360 27155 leader_election.cc:290] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:58:11.570425 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
W20260810 01:58:11.570691 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
I20260810 01:58:11.570824 24109 leader_election.cc:304] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:11.571192 27155 raft_consensus.cc:2749] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 01:58:13.029721 23889 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Couldn't send request to peer 50d44445234b4931aaa28141cdaa67ab. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 16: this message will repeat every 5th retry.
W20260810 01:58:13.673267 23892 consensus_peers.cc:597] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9b595b2ec2ff4503a20c1e9e7a7467b7. This is attempt 6: this message will repeat every 5th retry.
W20260810 01:58:13.949278 23892 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 16: this message will repeat every 5th retry.
I20260810 01:58:14.324200 27155 raft_consensus.cc:493] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:58:14.324409 27155 raft_consensus.cc:515] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:14.325068 27155 leader_election.cc:290] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:58:14.326040 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
W20260810 01:58:14.326293 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
I20260810 01:58:14.326390 24109 leader_election.cc:304] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:14.326697 27155 raft_consensus.cc:2749] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:58:14.569106 27155 raft_consensus.cc:493] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:58:14.569299 27155 raft_consensus.cc:515] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:14.569955 27155 leader_election.cc:290] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:58:14.571197 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
W20260810 01:58:14.571449 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
I20260810 01:58:14.571590 24109 leader_election.cc:304] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:14.571895 27155 raft_consensus.cc:2749] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:58:15.087359 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 01:58:15.087592 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 01:58:16.337677 27199 raft_consensus.cc:493] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 01:58:16.337890 27199 raft_consensus.cc:515] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:16.338613 27199 leader_election.cc:290] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 01:58:16.339645 23889 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
W20260810 01:58:16.339859 23892 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
I20260810 01:58:16.340027 23892 leader_election.cc:304] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:16.340303 27199 raft_consensus.cc:2749] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 01:58:17.107657 27290 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 01:58:17.107852 27290 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:58:17.108639 27290 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:58:17.109869 26771 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260810 01:58:17.111462 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 01:58:17.111661 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:58:17.111766 26771 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 01:58:17.112003 27290 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260810 01:58:19.303267 27307 raft_consensus.cc:493] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:58:19.303537 27307 raft_consensus.cc:515] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:19.304400 27307 leader_election.cc:290] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:58:19.305469 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
W20260810 01:58:19.305704 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
I20260810 01:58:19.305806 24109 leader_election.cc:304] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:19.306114 27307 raft_consensus.cc:2749] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:58:20.090741 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 01:58:20.090938 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 01:58:20.780804 23684 consensus_peers.cc:597] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f88158dec2a449c2a2011b8fc042f0ac. This is attempt 11: this message will repeat every 5th retry.
I20260810 01:58:22.779654 27388 raft_consensus.cc:493] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 01:58:22.779956 27388 raft_consensus.cc:515] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:22.780745 27388 leader_election.cc:290] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 01:58:22.781958 23889 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
W20260810 01:58:22.782210 23892 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
I20260810 01:58:22.782346 23892 leader_election.cc:304] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:22.782697 27388 raft_consensus.cc:2749] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 01:58:22.919831 27367 raft_consensus.cc:493] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:58:22.920023 27367 raft_consensus.cc:515] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:22.920627 27367 leader_election.cc:290] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:58:22.921727 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
W20260810 01:58:22.921977 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
I20260810 01:58:22.922114 24109 leader_election.cc:304] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:22.922391 27367 raft_consensus.cc:2749] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 01:58:23.041555 23892 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 21: this message will repeat every 5th retry.
I20260810 01:58:23.223605 27388 raft_consensus.cc:493] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 01:58:23.223850 27388 raft_consensus.cc:515] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:58:23.224982 27388 leader_election.cc:290] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 01:58:23.226109 23889 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
W20260810 01:58:23.226302 23892 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
I20260810 01:58:23.226486 23892 leader_election.cc:304] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:23.226774 27388 raft_consensus.cc:2749] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 01:58:23.403700 23889 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Couldn't send request to peer 50d44445234b4931aaa28141cdaa67ab. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 21: this message will repeat every 5th retry.
W20260810 01:58:23.949330 23892 consensus_peers.cc:597] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9b595b2ec2ff4503a20c1e9e7a7467b7. This is attempt 11: this message will repeat every 5th retry.
I20260810 01:58:25.093916 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 01:58:25.094110 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 01:58:25.780591 27367 raft_consensus.cc:493] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:58:25.780853 27367 raft_consensus.cc:515] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:25.781582 27367 leader_election.cc:290] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:58:25.782578 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
W20260810 01:58:25.782843 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
I20260810 01:58:25.782967 24109 leader_election.cc:304] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:25.783300 27367 raft_consensus.cc:2749] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:58:26.090307 27367 raft_consensus.cc:493] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:58:26.090539 27367 raft_consensus.cc:515] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:26.091408 27367 leader_election.cc:290] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:58:26.092267 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
W20260810 01:58:26.092538 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
I20260810 01:58:26.092677 24109 leader_election.cc:304] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:26.092984 27367 raft_consensus.cc:2749] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:58:26.943897 27367 raft_consensus.cc:493] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:58:26.944126 27367 raft_consensus.cc:515] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:26.944826 27367 leader_election.cc:290] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:58:26.946079 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
W20260810 01:58:26.946367 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
I20260810 01:58:26.946503 24109 leader_election.cc:304] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:26.946815 27367 raft_consensus.cc:2749] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:58:27.069769 27451 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 01:58:27.070032 27451 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:58:27.070833 27451 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:58:27.072273 26771 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260810 01:58:27.073774 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 01:58:27.074013 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:58:27.074189 26771 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 01:58:27.074481 27451 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260810 01:58:29.243608 27480 raft_consensus.cc:493] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 01:58:29.243885 27480 raft_consensus.cc:515] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:58:29.244637 27480 leader_election.cc:290] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 01:58:29.245884 23889 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
W20260810 01:58:29.246083 23892 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
I20260810 01:58:29.246206 23892 leader_election.cc:304] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:29.246515 27480 raft_consensus.cc:2749] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 01:58:30.008787 24301 heartbeater.cc:646] Failed to heartbeat to 127.22.69.61:41945 (60 consecutive failures): Network error: Failed to ping master at 127.22.69.61:41945: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
I20260810 01:58:30.097601 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 01:58:30.097832 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 01:58:30.204308 23841 heartbeater.cc:646] Failed to heartbeat to 127.22.69.60:36917 (113 consecutive failures): Network error: Failed to ping master at 127.22.69.60:36917: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
W20260810 01:58:30.316987 24069 heartbeater.cc:646] Failed to heartbeat to 127.22.69.61:41945 (109 consecutive failures): Network error: Failed to ping master at 127.22.69.61:41945: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 01:58:30.758106 23684 consensus_peers.cc:597] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f88158dec2a449c2a2011b8fc042f0ac. This is attempt 16: this message will repeat every 5th retry.
W20260810 01:58:32.603645 23892 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 26: this message will repeat every 5th retry.
I20260810 01:58:32.707998 27537 raft_consensus.cc:493] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:58:32.708251 27537 raft_consensus.cc:515] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:32.709019 27537 leader_election.cc:290] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:58:32.710191 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
W20260810 01:58:32.710386 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
I20260810 01:58:32.710508 24109 leader_election.cc:304] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:32.710816 27537 raft_consensus.cc:2749] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 01:58:33.423547 23892 consensus_peers.cc:597] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9b595b2ec2ff4503a20c1e9e7a7467b7. This is attempt 16: this message will repeat every 5th retry.
I20260810 01:58:33.623925 27562 raft_consensus.cc:493] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 01:58:33.624154 27562 raft_consensus.cc:515] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:33.624895 27562 leader_election.cc:290] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 01:58:33.625847 23889 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
W20260810 01:58:33.626034 23892 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
I20260810 01:58:33.626157 23892 leader_election.cc:304] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:33.626386 27562 raft_consensus.cc:2749] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 01:58:33.690469 23889 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Couldn't send request to peer 50d44445234b4931aaa28141cdaa67ab. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 26: this message will repeat every 5th retry.
I20260810 01:58:33.934839 27537 raft_consensus.cc:493] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:58:33.935034 27537 raft_consensus.cc:515] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:33.935873 27537 leader_election.cc:290] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:58:33.937135 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
W20260810 01:58:33.937402 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
I20260810 01:58:33.937500 24109 leader_election.cc:304] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:33.937829 27537 raft_consensus.cc:2749] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:58:34.019994 27537 raft_consensus.cc:493] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:58:34.020182 27537 raft_consensus.cc:515] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:34.020844 27537 leader_election.cc:290] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:58:34.021729 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
W20260810 01:58:34.021986 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
I20260810 01:58:34.022138 24109 leader_election.cc:304] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:34.022451 27537 raft_consensus.cc:2749] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:58:35.101114 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 01:58:35.101327 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 01:58:35.237758 27537 raft_consensus.cc:493] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:58:35.238013 27537 raft_consensus.cc:515] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:35.238904 27537 leader_election.cc:290] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:58:35.240152 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
W20260810 01:58:35.240407 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
I20260810 01:58:35.240525 24109 leader_election.cc:304] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:35.240831 27537 raft_consensus.cc:2749] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:58:36.252439 27583 raft_consensus.cc:493] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 01:58:36.252671 27583 raft_consensus.cc:515] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:58:36.253391 27583 leader_election.cc:290] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 01:58:36.254549 23889 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
W20260810 01:58:36.254751 23892 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
I20260810 01:58:36.254878 23892 leader_election.cc:304] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:36.255146 27583 raft_consensus.cc:2749] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 01:58:36.867918 27537 raft_consensus.cc:493] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:58:36.868113 27537 raft_consensus.cc:515] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:36.868765 27537 leader_election.cc:290] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:58:36.869908 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
W20260810 01:58:36.870110 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
I20260810 01:58:36.870203 24109 leader_election.cc:304] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:36.870466 27537 raft_consensus.cc:2749] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:58:40.104803 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 01:58:40.105027 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 01:58:40.874344 23684 consensus_peers.cc:597] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f88158dec2a449c2a2011b8fc042f0ac. This is attempt 21: this message will repeat every 5th retry.
I20260810 01:58:41.321941 27661 raft_consensus.cc:493] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:58:41.322111 27661 raft_consensus.cc:515] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:41.322688 27661 leader_election.cc:290] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:58:41.323769 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
W20260810 01:58:41.323979 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
I20260810 01:58:41.324075 24109 leader_election.cc:304] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:41.324299 27661 raft_consensus.cc:2749] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:58:41.362993 27672 raft_consensus.cc:493] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 01:58:41.363250 27672 raft_consensus.cc:515] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:58:41.364246 27672 leader_election.cc:290] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 01:58:41.365279 23889 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
W20260810 01:58:41.365464 23892 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
I20260810 01:58:41.365584 23892 leader_election.cc:304] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:41.365814 27672 raft_consensus.cc:2749] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 01:58:41.783505 27661 raft_consensus.cc:493] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:58:41.783746 27661 raft_consensus.cc:515] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:41.784582 27661 leader_election.cc:290] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:58:41.785647 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
W20260810 01:58:41.785874 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
I20260810 01:58:41.785984 24109 leader_election.cc:304] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:41.786278 27661 raft_consensus.cc:2749] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:58:42.122347 27672 raft_consensus.cc:493] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 01:58:42.122630 27672 raft_consensus.cc:515] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:42.123406 27672 leader_election.cc:290] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 01:58:42.124476 23892 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
W20260810 01:58:42.124680 23889 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
I20260810 01:58:42.124852 23889 leader_election.cc:304] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:42.125129 27672 raft_consensus.cc:2749] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 01:58:42.481814 23892 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 31: this message will repeat every 5th retry.
W20260810 01:58:43.235795 23892 consensus_peers.cc:597] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9b595b2ec2ff4503a20c1e9e7a7467b7. This is attempt 21: this message will repeat every 5th retry.
W20260810 01:58:44.294075 23889 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Couldn't send request to peer 50d44445234b4931aaa28141cdaa67ab. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 31: this message will repeat every 5th retry.
I20260810 01:58:45.108027 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 01:58:45.108253 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 01:58:45.130190 27737 raft_consensus.cc:493] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:58:45.130437 27737 raft_consensus.cc:515] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:45.131528 27737 leader_election.cc:290] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:58:45.132464 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
W20260810 01:58:45.132735 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
I20260810 01:58:45.132823 24109 leader_election.cc:304] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:45.133145 27737 raft_consensus.cc:2749] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:58:46.248093 27778 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 01:58:46.248354 27778 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:58:46.249477 27778 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:58:46.251370 26771 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260810 01:58:46.252934 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 01:58:46.253150 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:58:46.253293 26771 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 01:58:46.253582 27778 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260810 01:58:48.643987 27814 raft_consensus.cc:493] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:58:48.644220 27814 raft_consensus.cc:515] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:48.645017 27814 leader_election.cc:290] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:58:48.645874 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
W20260810 01:58:48.646154 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
I20260810 01:58:48.646270 24109 leader_election.cc:304] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:48.646517 27814 raft_consensus.cc:2749] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:58:50.062609 27839 raft_consensus.cc:493] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 01:58:50.062829 27839 raft_consensus.cc:515] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:50.063602 27839 leader_election.cc:290] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 01:58:50.064500 23889 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
W20260810 01:58:50.064700 23892 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
I20260810 01:58:50.064851 23892 leader_election.cc:304] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:50.065152 27839 raft_consensus.cc:2749] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 01:58:50.111343 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 01:58:50.111557 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 01:58:50.502722 23684 consensus_peers.cc:597] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f88158dec2a449c2a2011b8fc042f0ac. This is attempt 26: this message will repeat every 5th retry.
I20260810 01:58:50.678849 27814 raft_consensus.cc:493] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:58:50.679086 27814 raft_consensus.cc:515] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:50.680091 27814 leader_election.cc:290] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:58:50.681173 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
W20260810 01:58:50.681456 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
I20260810 01:58:50.681636 24109 leader_election.cc:304] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:50.681943 27814 raft_consensus.cc:2749] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:58:51.786914 27814 raft_consensus.cc:493] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:58:51.787196 27814 raft_consensus.cc:515] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:51.788149 27814 leader_election.cc:290] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:58:51.788990 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
W20260810 01:58:51.789289 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
I20260810 01:58:51.789412 24109 leader_election.cc:304] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:51.789665 27814 raft_consensus.cc:2749] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 01:58:53.187775 23892 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 36: this message will repeat every 5th retry.
W20260810 01:58:53.534219 23892 consensus_peers.cc:597] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9b595b2ec2ff4503a20c1e9e7a7467b7. This is attempt 26: this message will repeat every 5th retry.
I20260810 01:58:54.144475 27814 raft_consensus.cc:493] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:58:54.144701 27814 raft_consensus.cc:515] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:54.145637 27814 leader_election.cc:290] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:58:54.146320 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
W20260810 01:58:54.146757 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
I20260810 01:58:54.146926 24109 leader_election.cc:304] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:54.147294 27814 raft_consensus.cc:2749] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 01:58:54.378432 23889 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Couldn't send request to peer 50d44445234b4931aaa28141cdaa67ab. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 36: this message will repeat every 5th retry.
I20260810 01:58:54.386029 27923 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 01:58:54.386210 27923 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:58:54.387107 27923 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:58:54.388242 26771 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260810 01:58:54.389745 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 01:58:54.389969 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:58:54.390066 26771 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 01:58:54.390293 27923 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260810 01:58:54.642647 27911 raft_consensus.cc:493] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 01:58:54.642851 27911 raft_consensus.cc:515] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:58:54.643572 27911 leader_election.cc:290] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 01:58:54.644678 23892 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
W20260810 01:58:54.644878 23889 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
I20260810 01:58:54.644994 23889 leader_election.cc:304] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:54.645236 27911 raft_consensus.cc:2749] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 01:58:55.114802 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 01:58:55.115029 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 01:58:56.141777 27949 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 01:58:56.142007 27949 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:58:56.142794 27949 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:58:56.145152 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 01:58:56.145380 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:58:56.145495 26771 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 01:58:56.145743 27949 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260810 01:58:57.615347 27970 raft_consensus.cc:493] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 01:58:57.615602 27970 raft_consensus.cc:515] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:58:57.616631 27970 leader_election.cc:290] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 01:58:57.617581 23892 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
W20260810 01:58:57.617754 23889 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
I20260810 01:58:57.617942 23889 leader_election.cc:304] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:57.618214 27970 raft_consensus.cc:2749] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 01:58:58.200778 27970 raft_consensus.cc:493] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 01:58:58.201303 27970 raft_consensus.cc:515] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:58:58.204614 27970 leader_election.cc:290] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 01:58:58.207682 23892 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
W20260810 01:58:58.208410 23889 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
I20260810 01:58:58.208923 23889 leader_election.cc:304] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:58:58.209286 27970 raft_consensus.cc:2749] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 01:59:00.118219 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 01:59:00.118394 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 01:59:00.317473 23684 consensus_peers.cc:597] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f88158dec2a449c2a2011b8fc042f0ac. This is attempt 31: this message will repeat every 5th retry.
I20260810 01:59:00.939254 28021 raft_consensus.cc:493] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:59:00.939456 28021 raft_consensus.cc:515] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:59:00.940194 28021 leader_election.cc:290] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:59:00.941285 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
W20260810 01:59:00.941614 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
I20260810 01:59:00.941789 24109 leader_election.cc:304] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:59:00.942137 28021 raft_consensus.cc:2749] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:59:02.229254 28021 raft_consensus.cc:493] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:59:02.229498 28021 raft_consensus.cc:515] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:59:02.230283 28021 leader_election.cc:290] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:59:02.231191 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
W20260810 01:59:02.231436 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
I20260810 01:59:02.231549 24109 leader_election.cc:304] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:59:02.231833 28021 raft_consensus.cc:2749] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 01:59:02.481446 23892 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 41: this message will repeat every 5th retry.
I20260810 01:59:03.104569 28021 raft_consensus.cc:493] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:59:03.104826 28021 raft_consensus.cc:515] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:59:03.105523 28021 leader_election.cc:290] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:59:03.106632 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
W20260810 01:59:03.106817 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
I20260810 01:59:03.106906 24109 leader_election.cc:304] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:59:03.107210 28021 raft_consensus.cc:2749] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 01:59:03.447675 23892 consensus_peers.cc:597] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9b595b2ec2ff4503a20c1e9e7a7467b7. This is attempt 31: this message will repeat every 5th retry.
W20260810 01:59:04.513403 23889 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Couldn't send request to peer 50d44445234b4931aaa28141cdaa67ab. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 41: this message will repeat every 5th retry.
I20260810 01:59:05.121981 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 01:59:05.122231 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 01:59:06.586431 28021 raft_consensus.cc:493] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:59:06.586664 28021 raft_consensus.cc:515] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:59:06.587643 28021 leader_election.cc:290] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:59:06.588472 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
W20260810 01:59:06.588783 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
I20260810 01:59:06.588953 24109 leader_election.cc:304] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:59:06.589234 28021 raft_consensus.cc:2749] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:59:08.840415 28152 raft_consensus.cc:493] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:59:08.840710 28152 raft_consensus.cc:515] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:59:08.841432 28152 leader_election.cc:290] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:59:08.842590 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
W20260810 01:59:08.842831 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
I20260810 01:59:08.842995 24109 leader_election.cc:304] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:59:08.843283 28152 raft_consensus.cc:2749] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:59:08.890053 28181 raft_consensus.cc:493] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 01:59:08.890276 28181 raft_consensus.cc:515] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:59:08.891049 28181 leader_election.cc:290] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 01:59:08.892007 23889 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
W20260810 01:59:08.892164 23892 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
I20260810 01:59:08.892288 23892 leader_election.cc:304] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:59:08.892571 28181 raft_consensus.cc:2749] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 01:59:10.125324 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 01:59:10.125571 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 01:59:10.723774 23684 consensus_peers.cc:597] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f88158dec2a449c2a2011b8fc042f0ac. This is attempt 36: this message will repeat every 5th retry.
W20260810 01:59:12.203260 23892 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 46: this message will repeat every 5th retry.
W20260810 01:59:13.587121 23892 consensus_peers.cc:597] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9b595b2ec2ff4503a20c1e9e7a7467b7. This is attempt 36: this message will repeat every 5th retry.
W20260810 01:59:13.739187 23889 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Couldn't send request to peer 50d44445234b4931aaa28141cdaa67ab. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 46: this message will repeat every 5th retry.
I20260810 01:59:14.525904 28296 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 01:59:14.526129 28296 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:59:14.527140 28296 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:59:14.528443 26771 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260810 01:59:14.529815 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 01:59:14.530055 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:59:14.530151 26771 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 01:59:14.530386 28296 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260810 01:59:15.128183 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 01:59:15.128363 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 01:59:16.567636 28322 raft_consensus.cc:493] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 01:59:16.567864 28322 raft_consensus.cc:515] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:59:16.568799 28322 leader_election.cc:290] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 01:59:16.569656 23889 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
W20260810 01:59:16.569841 23892 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
I20260810 01:59:16.569969 23892 leader_election.cc:304] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:59:16.570207 28322 raft_consensus.cc:2749] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 01:59:16.662184 28152 raft_consensus.cc:493] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:59:16.662456 28152 raft_consensus.cc:515] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:59:16.663241 28152 leader_election.cc:290] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:59:16.664145 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
W20260810 01:59:16.664454 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
I20260810 01:59:16.664547 24109 leader_election.cc:304] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:59:16.664856 28152 raft_consensus.cc:2749] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:59:17.292858 28152 raft_consensus.cc:493] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:59:17.293103 28152 raft_consensus.cc:515] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:59:17.293887 28152 leader_election.cc:290] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:59:17.295131 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
W20260810 01:59:17.295432 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
I20260810 01:59:17.295644 24109 leader_election.cc:304] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:59:17.295972 28152 raft_consensus.cc:2749] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:59:18.194489 28152 raft_consensus.cc:493] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:59:18.194749 28152 raft_consensus.cc:515] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:59:18.195652 28152 leader_election.cc:290] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:59:18.196693 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
W20260810 01:59:18.196961 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
I20260810 01:59:18.197094 24109 leader_election.cc:304] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:59:18.197412 28152 raft_consensus.cc:2749] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:59:18.246848 28152 raft_consensus.cc:493] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:59:18.247128 28152 raft_consensus.cc:515] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:59:18.247820 28152 leader_election.cc:290] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:59:18.248826 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
W20260810 01:59:18.249086 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
I20260810 01:59:18.249220 24109 leader_election.cc:304] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:59:18.249542 28152 raft_consensus.cc:2749] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:59:20.132771 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 01:59:20.132982 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 01:59:20.526899 23684 consensus_peers.cc:597] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f88158dec2a449c2a2011b8fc042f0ac. This is attempt 41: this message will repeat every 5th retry.
I20260810 01:59:21.245779 28391 raft_consensus.cc:493] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 01:59:21.246025 28391 raft_consensus.cc:515] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:59:21.246913 28391 leader_election.cc:290] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 01:59:21.247956 23889 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
W20260810 01:59:21.248145 23892 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
I20260810 01:59:21.248270 23892 leader_election.cc:304] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:59:21.248507 28391 raft_consensus.cc:2749] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 01:59:21.560890 23892 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 51: this message will repeat every 5th retry.
I20260810 01:59:22.420521 28431 raft_consensus.cc:493] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:59:22.420734 28431 raft_consensus.cc:515] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:59:22.421665 28431 leader_election.cc:290] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:59:22.422905 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
W20260810 01:59:22.423183 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
I20260810 01:59:22.423303 24109 leader_election.cc:304] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:59:22.423610 28431 raft_consensus.cc:2749] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 01:59:23.500049 23889 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Couldn't send request to peer 50d44445234b4931aaa28141cdaa67ab. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 51: this message will repeat every 5th retry.
W20260810 01:59:23.658036 23892 consensus_peers.cc:597] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9b595b2ec2ff4503a20c1e9e7a7467b7. This is attempt 41: this message will repeat every 5th retry.
I20260810 01:59:23.799897 28459 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 01:59:23.800096 28459 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:59:23.800900 28459 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:59:23.802423 26771 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260810 01:59:23.804381 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 01:59:23.804651 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:59:23.804783 26771 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 01:59:23.805051 28459 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260810 01:59:25.135968 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 01:59:25.136184 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 01:59:30.139128 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 01:59:30.139345 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 01:59:30.160527 24301 heartbeater.cc:646] Failed to heartbeat to 127.22.69.61:41945 (180 consecutive failures): Network error: Failed to ping master at 127.22.69.61:41945: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 01:59:30.406509 24063 heartbeater.cc:646] Failed to heartbeat to 127.22.69.60:36917 (399 consecutive failures): Network error: Failed to ping master at 127.22.69.60:36917: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
W20260810 01:59:30.500036 23841 heartbeater.cc:646] Failed to heartbeat to 127.22.69.60:36917 (264 consecutive failures): Network error: Failed to ping master at 127.22.69.60:36917: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
W20260810 01:59:30.752631 23684 consensus_peers.cc:597] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f88158dec2a449c2a2011b8fc042f0ac. This is attempt 46: this message will repeat every 5th retry.
I20260810 01:59:31.643127 28616 raft_consensus.cc:493] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:59:31.643323 28616 raft_consensus.cc:515] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:59:31.644037 28616 leader_election.cc:290] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:59:31.645275 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
W20260810 01:59:31.645516 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
I20260810 01:59:31.645654 24109 leader_election.cc:304] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:59:31.645943 28616 raft_consensus.cc:2749] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 01:59:31.655391 23892 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 56: this message will repeat every 5th retry.
I20260810 01:59:32.728120 28616 raft_consensus.cc:493] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:59:32.728382 28616 raft_consensus.cc:515] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:59:32.729111 28616 leader_election.cc:290] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:59:32.730216 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
W20260810 01:59:32.730427 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
I20260810 01:59:32.730526 24109 leader_election.cc:304] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:59:32.730765 28616 raft_consensus.cc:2749] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:59:32.982158 28657 raft_consensus.cc:493] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 01:59:32.982352 28657 raft_consensus.cc:515] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:59:32.983207 28657 leader_election.cc:290] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 01:59:32.984162 23889 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
W20260810 01:59:32.984344 23892 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
I20260810 01:59:32.984455 23892 leader_election.cc:304] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:59:32.984673 28657 raft_consensus.cc:2749] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 01:59:33.175380 23889 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Couldn't send request to peer 50d44445234b4931aaa28141cdaa67ab. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 56: this message will repeat every 5th retry.
W20260810 01:59:33.735020 23892 consensus_peers.cc:597] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9b595b2ec2ff4503a20c1e9e7a7467b7. This is attempt 46: this message will repeat every 5th retry.
I20260810 01:59:34.744704 28689 raft_consensus.cc:493] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 01:59:34.744935 28689 raft_consensus.cc:515] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:59:34.745801 28689 leader_election.cc:290] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 01:59:34.746866 23889 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
W20260810 01:59:34.747064 23892 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
I20260810 01:59:34.747259 23892 leader_election.cc:304] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:59:34.747566 28689 raft_consensus.cc:2749] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 01:59:34.965839 28702 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 01:59:34.966061 28702 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:59:34.967203 28702 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:59:34.968475 26771 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260810 01:59:34.969973 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 01:59:34.970222 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:59:34.970331 26771 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 01:59:34.970594 28702 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260810 01:59:35.142057 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 01:59:35.142313 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 01:59:36.061266 28616 raft_consensus.cc:493] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:59:36.061550 28616 raft_consensus.cc:515] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:59:36.062266 28616 leader_election.cc:290] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:59:36.063313 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
W20260810 01:59:36.063541 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
I20260810 01:59:36.063664 24109 leader_election.cc:304] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:59:36.063930 28616 raft_consensus.cc:2749] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:59:36.586236 28616 raft_consensus.cc:493] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:59:36.586477 28616 raft_consensus.cc:515] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:59:36.587239 28616 leader_election.cc:290] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:59:36.588164 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
W20260810 01:59:36.588449 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
I20260810 01:59:36.588568 24109 leader_election.cc:304] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:59:36.588861 28616 raft_consensus.cc:2749] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:59:37.437717 28616 raft_consensus.cc:493] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:59:37.437981 28616 raft_consensus.cc:515] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:59:37.439049 28616 leader_election.cc:290] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:59:37.440091 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
W20260810 01:59:37.440357 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
I20260810 01:59:37.440508 24109 leader_election.cc:304] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:59:37.440858 28616 raft_consensus.cc:2749] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 01:59:39.915199 23684 consensus_peers.cc:597] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f88158dec2a449c2a2011b8fc042f0ac. This is attempt 51: this message will repeat every 5th retry.
I20260810 01:59:39.943995 28777 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 01:59:39.944293 28777 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:59:39.945252 28777 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:59:39.947950 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 01:59:39.948189 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:59:39.948333 26771 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 01:59:39.948565 28777 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260810 01:59:40.145330 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 01:59:40.145527 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 01:59:41.714267 28803 raft_consensus.cc:493] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:59:41.714483 28803 raft_consensus.cc:515] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:59:41.715193 28803 leader_election.cc:290] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:59:41.716223 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
W20260810 01:59:41.716540 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
I20260810 01:59:41.716661 24109 leader_election.cc:304] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:59:41.716935 28803 raft_consensus.cc:2749] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 01:59:41.898386 23892 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 61: this message will repeat every 5th retry.
W20260810 01:59:42.780196 23892 consensus_peers.cc:597] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9b595b2ec2ff4503a20c1e9e7a7467b7. This is attempt 51: this message will repeat every 5th retry.
W20260810 01:59:43.377483 23889 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Couldn't send request to peer 50d44445234b4931aaa28141cdaa67ab. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 61: this message will repeat every 5th retry.
I20260810 01:59:43.550917 28841 raft_consensus.cc:493] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:59:43.551221 28841 raft_consensus.cc:515] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:59:43.551970 28841 leader_election.cc:290] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:59:43.553087 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
W20260810 01:59:43.553371 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
I20260810 01:59:43.553520 24109 leader_election.cc:304] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:59:43.553805 28841 raft_consensus.cc:2749] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:59:43.695706 28789 raft_consensus.cc:493] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 01:59:43.695899 28789 raft_consensus.cc:515] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:59:43.696599 28789 leader_election.cc:290] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 01:59:43.697553 23889 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
W20260810 01:59:43.697734 23892 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
I20260810 01:59:43.697851 23892 leader_election.cc:304] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:59:43.698096 28789 raft_consensus.cc:2749] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 01:59:44.550237 28850 raft_consensus.cc:493] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:59:44.550518 28850 raft_consensus.cc:515] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:59:44.551347 28850 leader_election.cc:290] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:59:44.552488 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
W20260810 01:59:44.552778 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
I20260810 01:59:44.552891 24109 leader_election.cc:304] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:59:44.553112 28850 raft_consensus.cc:2749] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:59:45.148255 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 01:59:45.148460 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 01:59:46.642033 28892 raft_consensus.cc:493] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 01:59:46.642228 28892 raft_consensus.cc:515] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:59:46.642984 28892 leader_election.cc:290] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 01:59:46.644044 23892 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
W20260810 01:59:46.644254 23889 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
I20260810 01:59:46.644425 23889 leader_election.cc:304] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:59:46.644696 28892 raft_consensus.cc:2749] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 01:59:47.783861 28922 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 01:59:47.784065 28922 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:59:47.784801 28922 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:59:47.786237 26771 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260810 01:59:47.788614 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 01:59:47.788875 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:59:47.789053 26771 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 01:59:47.789366 28922 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260810 01:59:48.391793 28921 raft_consensus.cc:493] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:59:48.392048 28921 raft_consensus.cc:515] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:59:48.392753 28921 leader_election.cc:290] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:59:48.394048 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
W20260810 01:59:48.394228 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
I20260810 01:59:48.394316 24109 leader_election.cc:304] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:59:48.394588 28921 raft_consensus.cc:2749] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:59:48.922132 28921 raft_consensus.cc:493] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:59:48.922360 28921 raft_consensus.cc:515] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:59:48.923026 28921 leader_election.cc:290] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:59:48.923801 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
W20260810 01:59:48.924126 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
I20260810 01:59:48.924216 24109 leader_election.cc:304] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:59:48.924500 28921 raft_consensus.cc:2749] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:59:49.048542 28921 raft_consensus.cc:493] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:59:49.048761 28921 raft_consensus.cc:515] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:59:49.049535 28921 leader_election.cc:290] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:59:49.050546 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
W20260810 01:59:49.050789 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
I20260810 01:59:49.050920 24109 leader_election.cc:304] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:59:49.051218 28921 raft_consensus.cc:2749] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:59:49.337601 28921 raft_consensus.cc:493] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:59:49.337800 28921 raft_consensus.cc:515] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:59:49.338497 28921 leader_election.cc:290] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:59:49.339505 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
W20260810 01:59:49.339840 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
I20260810 01:59:49.340000 24109 leader_election.cc:304] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:59:49.340307 28921 raft_consensus.cc:2749] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 01:59:49.700018 23684 consensus_peers.cc:597] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f88158dec2a449c2a2011b8fc042f0ac. This is attempt 56: this message will repeat every 5th retry.
I20260810 01:59:50.151676 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 01:59:50.151870 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 01:59:51.194900 23892 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 66: this message will repeat every 5th retry.
W20260810 01:59:52.423938 23892 consensus_peers.cc:597] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9b595b2ec2ff4503a20c1e9e7a7467b7. This is attempt 56: this message will repeat every 5th retry.
W20260810 01:59:53.198557 23889 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Couldn't send request to peer 50d44445234b4931aaa28141cdaa67ab. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 66: this message will repeat every 5th retry.
I20260810 01:59:53.247925 29017 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 01:59:53.248167 29017 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 01:59:53.248876 29017 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 01:59:53.250226 26771 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260810 01:59:53.251765 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 01:59:53.251988 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 01:59:53.252090 26771 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 01:59:53.252332 29017 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260810 01:59:55.155053 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 01:59:55.155288 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 01:59:55.455760 29055 raft_consensus.cc:493] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:59:55.455996 29055 raft_consensus.cc:515] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:59:55.457019 29055 leader_election.cc:290] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:59:55.458077 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
W20260810 01:59:55.458343 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
I20260810 01:59:55.458487 24109 leader_election.cc:304] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:59:55.458757 29055 raft_consensus.cc:2749] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:59:58.115109 29055 raft_consensus.cc:493] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:59:58.115312 29055 raft_consensus.cc:515] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:59:58.116209 29055 leader_election.cc:290] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
I20260810 01:59:58.117053 29090 raft_consensus.cc:493] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 01:59:58.117244 29090 raft_consensus.cc:515] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
W20260810 01:59:58.117532 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
W20260810 01:59:58.117833 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
I20260810 01:59:58.117995 24109 leader_election.cc:304] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:59:58.118170 29090 leader_election.cc:290] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
I20260810 01:59:58.118463 29055 raft_consensus.cc:2749] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 01:59:58.118880 23889 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
W20260810 01:59:58.119266 23892 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
I20260810 01:59:58.119406 23892 leader_election.cc:304] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:59:58.119627 29090 raft_consensus.cc:2749] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 01:59:58.401952 29055 raft_consensus.cc:493] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:59:58.402223 29055 raft_consensus.cc:515] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:59:58.402937 29055 leader_election.cc:290] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:59:58.404183 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
W20260810 01:59:58.404371 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
I20260810 01:59:58.404462 24109 leader_election.cc:304] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:59:58.404732 29055 raft_consensus.cc:2749] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 01:59:58.466712 29110 raft_consensus.cc:493] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 01:59:58.466987 29110 raft_consensus.cc:515] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 01:59:58.467926 29110 leader_election.cc:290] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 01:59:58.468961 23889 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
W20260810 01:59:58.469137 23892 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
I20260810 01:59:58.469288 23892 leader_election.cc:304] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:59:58.469568 29110 raft_consensus.cc:2749] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 01:59:58.501516 29055 raft_consensus.cc:493] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 01:59:58.501716 29055 raft_consensus.cc:515] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 01:59:58.502476 29055 leader_election.cc:290] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 01:59:58.503520 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
W20260810 01:59:58.503826 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
I20260810 01:59:58.503959 24109 leader_election.cc:304] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 01:59:58.504233 29055 raft_consensus.cc:2749] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 01:59:59.768908 23684 consensus_peers.cc:597] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f88158dec2a449c2a2011b8fc042f0ac. This is attempt 61: this message will repeat every 5th retry.
I20260810 02:00:00.068615 26781 maintenance_manager.cc:419] P 919313c0dee540a690f46dbc8208d20b: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033352
I20260810 02:00:00.128350 26780 maintenance_manager.cc:643] P 919313c0dee540a690f46dbc8208d20b: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.059s user 0.028s sys 0.003s Metrics: {"bytes_written":15759,"cfile_init":1,"dirs.queue_time_us":468,"dirs.run_cpu_time_us":291,"dirs.run_wall_time_us":1363,"drs_written":1,"lbm_read_time_us":67,"lbm_reads_lt_1ms":4,"lbm_write_time_us":697,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":35,"thread_start_us":551,"threads_started":2,"wal-append.queue_time_us":305}
I20260810 02:00:00.129676 26781 maintenance_manager.cc:419] P 919313c0dee540a690f46dbc8208d20b: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1729 bytes on disk
I20260810 02:00:00.130422 26780 maintenance_manager.cc:643] P 919313c0dee540a690f46dbc8208d20b: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":68,"lbm_reads_lt_1ms":4}
I20260810 02:00:00.158308 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 02:00:00.158486 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 02:00:01.202253 29151 raft_consensus.cc:493] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:00:01.202597 29151 raft_consensus.cc:515] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:00:01.203365 29151 leader_election.cc:290] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:00:01.204631 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
W20260810 02:00:01.204872 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
I20260810 02:00:01.205009 24109 leader_election.cc:304] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:00:01.205302 29151 raft_consensus.cc:2749] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 02:00:01.333063 23892 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 71: this message will repeat every 5th retry.
W20260810 02:00:02.225713 23892 consensus_peers.cc:597] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9b595b2ec2ff4503a20c1e9e7a7467b7. This is attempt 61: this message will repeat every 5th retry.
I20260810 02:00:02.653559 29187 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 02:00:02.653825 29187 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 02:00:02.654599 29187 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 02:00:02.656030 26771 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260810 02:00:02.664332 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 02:00:02.664597 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 02:00:02.664816 26771 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 02:00:02.665127 29187 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260810 02:00:03.433228 23889 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Couldn't send request to peer 50d44445234b4931aaa28141cdaa67ab. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 71: this message will repeat every 5th retry.
I20260810 02:00:05.161224 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 02:00:05.161495 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 02:00:06.023128 29182 raft_consensus.cc:493] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:00:06.023327 29182 raft_consensus.cc:515] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:00:06.024056 29182 leader_election.cc:290] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:00:06.025254 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
W20260810 02:00:06.025607 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
I20260810 02:00:06.025777 24109 leader_election.cc:304] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:00:06.026087 29182 raft_consensus.cc:2749] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:00:10.164431 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 02:00:10.164690 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 02:00:10.361863 23684 consensus_peers.cc:597] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f88158dec2a449c2a2011b8fc042f0ac. This is attempt 66: this message will repeat every 5th retry.
I20260810 02:00:11.019477 29362 raft_consensus.cc:493] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:00:11.019752 29362 raft_consensus.cc:515] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:00:11.020714 29362 leader_election.cc:290] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:00:11.021672 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
W20260810 02:00:11.021893 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
I20260810 02:00:11.022022 24109 leader_election.cc:304] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:00:11.022267 29362 raft_consensus.cc:2749] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 02:00:11.612918 23892 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 76: this message will repeat every 5th retry.
I20260810 02:00:12.037462 29403 raft_consensus.cc:493] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 02:00:12.037815 29403 raft_consensus.cc:515] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:00:12.038784 29403 leader_election.cc:290] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 02:00:12.039937 23889 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
W20260810 02:00:12.040139 23892 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
I20260810 02:00:12.040307 23892 leader_election.cc:304] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:00:12.040644 29403 raft_consensus.cc:2749] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 02:00:12.289359 29362 raft_consensus.cc:493] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:00:12.289592 29362 raft_consensus.cc:515] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:00:12.290364 29362 leader_election.cc:290] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:00:12.291226 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
W20260810 02:00:12.291519 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
I20260810 02:00:12.291643 24109 leader_election.cc:304] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:00:12.291935 29362 raft_consensus.cc:2749] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 02:00:12.487831 23889 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Couldn't send request to peer 50d44445234b4931aaa28141cdaa67ab. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 76: this message will repeat every 5th retry.
W20260810 02:00:12.819739 23892 consensus_peers.cc:597] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9b595b2ec2ff4503a20c1e9e7a7467b7. This is attempt 66: this message will repeat every 5th retry.
I20260810 02:00:13.750715 29404 raft_consensus.cc:493] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 02:00:13.750977 29404 raft_consensus.cc:515] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 02:00:13.752024 29404 leader_election.cc:290] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 02:00:13.752941 23889 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
W20260810 02:00:13.753183 23892 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
I20260810 02:00:13.753346 23892 leader_election.cc:304] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:00:13.753684 29404 raft_consensus.cc:2749] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 02:00:15.167977 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 02:00:15.168289 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 02:00:17.692914 29469 raft_consensus.cc:493] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:00:17.693109 29469 raft_consensus.cc:515] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:00:17.693852 29469 leader_election.cc:290] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:00:17.695066 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
W20260810 02:00:17.695432 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
I20260810 02:00:17.695587 24109 leader_election.cc:304] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:00:17.695931 29469 raft_consensus.cc:2749] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:00:17.743324 29469 raft_consensus.cc:493] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:00:17.743585 29469 raft_consensus.cc:515] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:00:17.744302 29469 leader_election.cc:290] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:00:17.745502 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
W20260810 02:00:17.745774 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
I20260810 02:00:17.745870 24109 leader_election.cc:304] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:00:17.746204 29469 raft_consensus.cc:2749] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:00:20.171705 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 02:00:20.171924 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 02:00:20.590610 23684 consensus_peers.cc:597] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f88158dec2a449c2a2011b8fc042f0ac. This is attempt 71: this message will repeat every 5th retry.
I20260810 02:00:20.741880 29588 raft_consensus.cc:493] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 02:00:20.742103 29588 raft_consensus.cc:515] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:00:20.743136 29588 leader_election.cc:290] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 02:00:20.744254 23892 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
W20260810 02:00:20.744503 23889 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
I20260810 02:00:20.744673 23889 leader_election.cc:304] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:00:20.744989 29588 raft_consensus.cc:2749] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 02:00:21.188017 23892 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 81: this message will repeat every 5th retry.
I20260810 02:00:21.565793 29600 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 02:00:21.566080 29600 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 02:00:21.567202 29600 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 02:00:21.568689 26771 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260810 02:00:21.572528 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 02:00:21.572767 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 02:00:21.572897 26771 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 02:00:21.573201 29600 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260810 02:00:22.384510 29469 raft_consensus.cc:493] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:00:22.384745 29469 raft_consensus.cc:515] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:00:22.385541 29469 leader_election.cc:290] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:00:22.386775 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
W20260810 02:00:22.387020 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
I20260810 02:00:22.387188 24109 leader_election.cc:304] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:00:22.387532 29469 raft_consensus.cc:2749] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 02:00:22.679706 23889 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Couldn't send request to peer 50d44445234b4931aaa28141cdaa67ab. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 81: this message will repeat every 5th retry.
W20260810 02:00:23.004760 23892 consensus_peers.cc:597] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9b595b2ec2ff4503a20c1e9e7a7467b7. This is attempt 71: this message will repeat every 5th retry.
I20260810 02:00:25.175356 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 02:00:25.175595 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 02:00:25.225216 29677 raft_consensus.cc:493] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:00:25.225447 29677 raft_consensus.cc:515] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:00:25.226249 29677 leader_election.cc:290] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:00:25.227452 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
W20260810 02:00:25.227731 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
I20260810 02:00:25.227859 24109 leader_election.cc:304] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:00:25.228219 29677 raft_consensus.cc:2749] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:00:25.572892 29677 raft_consensus.cc:493] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:00:25.573194 29677 raft_consensus.cc:515] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:00:25.574101 29677 leader_election.cc:290] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:00:25.575376 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
W20260810 02:00:25.575696 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
I20260810 02:00:25.575875 24109 leader_election.cc:304] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:00:25.576184 29677 raft_consensus.cc:2749] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:00:28.564292 29746 raft_consensus.cc:493] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 02:00:28.564553 29746 raft_consensus.cc:515] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 02:00:28.565337 29746 leader_election.cc:290] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 02:00:28.566421 23889 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
W20260810 02:00:28.566603 23892 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
I20260810 02:00:28.566814 23892 leader_election.cc:304] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:00:28.567152 29746 raft_consensus.cc:2749] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 02:00:29.405460 29793 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 02:00:29.405718 29793 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 02:00:29.406562 29793 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 02:00:29.407894 26771 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260810 02:00:29.409756 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 02:00:29.410035 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 02:00:29.410208 26771 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 02:00:29.410701 29793 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260810 02:00:30.042161 29792 raft_consensus.cc:493] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:00:30.042397 29792 raft_consensus.cc:515] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:00:30.043329 29792 leader_election.cc:290] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:00:30.044294 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
W20260810 02:00:30.044565 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
I20260810 02:00:30.044687 24109 leader_election.cc:304] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:00:30.044971 29792 raft_consensus.cc:2749] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:00:30.178941 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 02:00:30.179314 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 02:00:30.312727 24301 heartbeater.cc:646] Failed to heartbeat to 127.22.69.61:41945 (300 consecutive failures): Network error: Failed to ping master at 127.22.69.61:41945: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 02:00:30.506582 23841 heartbeater.cc:646] Failed to heartbeat to 127.22.69.60:36917 (415 consecutive failures): Network error: Failed to ping master at 127.22.69.60:36917: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
W20260810 02:00:30.507618 23684 consensus_peers.cc:597] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f88158dec2a449c2a2011b8fc042f0ac. This is attempt 76: this message will repeat every 5th retry.
W20260810 02:00:30.804021 24063 heartbeater.cc:646] Failed to heartbeat to 127.22.69.60:36917 (631 consecutive failures): Network error: Failed to ping master at 127.22.69.60:36917: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 02:00:30.837868 29816 raft_consensus.cc:493] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 02:00:30.838109 29816 raft_consensus.cc:515] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:00:30.839035 29816 leader_election.cc:290] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 02:00:30.840116 23889 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
W20260810 02:00:30.840314 23892 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
I20260810 02:00:30.840463 23892 leader_election.cc:304] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:00:30.840716 29816 raft_consensus.cc:2749] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 02:00:31.023260 23892 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 86: this message will repeat every 5th retry.
I20260810 02:00:31.284204 29792 raft_consensus.cc:493] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:00:31.284443 29792 raft_consensus.cc:515] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:00:31.285207 29792 leader_election.cc:290] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:00:31.286291 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
W20260810 02:00:31.286511 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
I20260810 02:00:31.286638 24109 leader_election.cc:304] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:00:31.286921 29792 raft_consensus.cc:2749] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:00:32.034799 29792 raft_consensus.cc:493] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:00:32.035104 29792 raft_consensus.cc:515] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:00:32.036013 29792 leader_election.cc:290] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:00:32.037155 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
W20260810 02:00:32.037432 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
I20260810 02:00:32.037533 24109 leader_election.cc:304] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:00:32.037853 29792 raft_consensus.cc:2749] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 02:00:32.219154 23889 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Couldn't send request to peer 50d44445234b4931aaa28141cdaa67ab. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 86: this message will repeat every 5th retry.
W20260810 02:00:32.741160 23892 consensus_peers.cc:597] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9b595b2ec2ff4503a20c1e9e7a7467b7. This is attempt 76: this message will repeat every 5th retry.
I20260810 02:00:35.182430 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 02:00:35.182677 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 02:00:35.779775 29792 raft_consensus.cc:493] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:00:35.780006 29792 raft_consensus.cc:515] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:00:35.780668 29792 leader_election.cc:290] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:00:35.781785 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
W20260810 02:00:35.782038 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
I20260810 02:00:35.782176 24109 leader_election.cc:304] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:00:35.782462 29792 raft_consensus.cc:2749] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:00:36.849113 29792 raft_consensus.cc:493] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:00:36.849315 29792 raft_consensus.cc:515] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:00:36.850019 29792 leader_election.cc:290] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:00:36.851056 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
W20260810 02:00:36.851325 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
I20260810 02:00:36.851409 24109 leader_election.cc:304] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:00:36.851631 29792 raft_consensus.cc:2749] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:00:37.181695 29931 raft_consensus.cc:493] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 02:00:37.181926 29931 raft_consensus.cc:515] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 02:00:37.182694 29931 leader_election.cc:290] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 02:00:37.183720 23889 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
W20260810 02:00:37.183895 23892 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
I20260810 02:00:37.184082 23892 leader_election.cc:304] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:00:37.184396 29931 raft_consensus.cc:2749] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 02:00:37.860671 29964 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 02:00:37.860946 29964 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 02:00:37.861680 29964 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 02:00:37.863123 26771 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260810 02:00:37.864658 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 02:00:37.864898 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 02:00:37.865027 26771 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 02:00:37.865290 29964 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260810 02:00:40.185755 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 02:00:40.185945 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 02:00:40.730023 23684 consensus_peers.cc:597] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f88158dec2a449c2a2011b8fc042f0ac. This is attempt 81: this message will repeat every 5th retry.
W20260810 02:00:40.965116 23892 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 91: this message will repeat every 5th retry.
W20260810 02:00:42.164312 23889 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Couldn't send request to peer 50d44445234b4931aaa28141cdaa67ab. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 91: this message will repeat every 5th retry.
I20260810 02:00:42.169837 29792 raft_consensus.cc:493] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:00:42.170039 29792 raft_consensus.cc:515] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:00:42.170670 29792 leader_election.cc:290] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:00:42.171613 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
W20260810 02:00:42.171792 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
I20260810 02:00:42.171893 24109 leader_election.cc:304] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:00:42.172112 29792 raft_consensus.cc:2749] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:00:42.798224 29792 raft_consensus.cc:493] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:00:42.798434 29792 raft_consensus.cc:515] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:00:42.799201 29792 leader_election.cc:290] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:00:42.800117 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
W20260810 02:00:42.800347 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
I20260810 02:00:42.800436 24109 leader_election.cc:304] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:00:42.800684 29792 raft_consensus.cc:2749] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 02:00:43.031992 23892 consensus_peers.cc:597] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9b595b2ec2ff4503a20c1e9e7a7467b7. This is attempt 81: this message will repeat every 5th retry.
I20260810 02:00:43.395998 30085 raft_consensus.cc:493] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 02:00:43.396242 30085 raft_consensus.cc:515] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 02:00:43.397042 30085 leader_election.cc:290] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 02:00:43.397953 23889 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
W20260810 02:00:43.398114 23892 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
I20260810 02:00:43.398243 23892 leader_election.cc:304] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:00:43.398468 30085 raft_consensus.cc:2749] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 02:00:44.779381 30099 raft_consensus.cc:493] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 02:00:44.779654 30099 raft_consensus.cc:515] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:00:44.780395 30099 leader_election.cc:290] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 02:00:44.781445 23889 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
W20260810 02:00:44.781630 23892 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
I20260810 02:00:44.781761 23892 leader_election.cc:304] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:00:44.782003 30099 raft_consensus.cc:2749] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 02:00:45.168742 29792 raft_consensus.cc:493] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:00:45.168984 29792 raft_consensus.cc:515] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:00:45.169827 29792 leader_election.cc:290] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:00:45.170848 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
W20260810 02:00:45.171126 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
I20260810 02:00:45.171236 24109 leader_election.cc:304] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:00:45.171480 29792 raft_consensus.cc:2749] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:00:45.188856 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 02:00:45.189040 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 02:00:46.744899 29792 raft_consensus.cc:493] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:00:46.745138 29792 raft_consensus.cc:515] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:00:46.745877 29792 leader_election.cc:290] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:00:46.746942 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
W20260810 02:00:46.747223 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
I20260810 02:00:46.747313 24109 leader_election.cc:304] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:00:46.747596 29792 raft_consensus.cc:2749] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:00:49.274034 29792 raft_consensus.cc:493] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:00:49.274273 29792 raft_consensus.cc:515] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:00:49.275162 29792 leader_election.cc:290] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:00:49.276262 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
W20260810 02:00:49.276527 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
I20260810 02:00:49.276670 24109 leader_election.cc:304] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:00:49.276954 29792 raft_consensus.cc:2749] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:00:50.192019 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 02:00:50.192286 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 02:00:50.250770 23684 consensus_peers.cc:597] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f88158dec2a449c2a2011b8fc042f0ac. This is attempt 86: this message will repeat every 5th retry.
I20260810 02:00:50.854409 30255 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 02:00:50.854672 30255 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 02:00:50.855537 30255 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 02:00:50.856763 26771 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260810 02:00:50.858178 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 02:00:50.858407 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 02:00:50.858506 26771 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 02:00:50.858740 30255 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260810 02:00:51.526562 23892 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 96: this message will repeat every 5th retry.
I20260810 02:00:51.537072 29792 raft_consensus.cc:493] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:00:51.537264 29792 raft_consensus.cc:515] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:00:51.537914 29792 leader_election.cc:290] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:00:51.538825 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
W20260810 02:00:51.539211 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
I20260810 02:00:51.539388 24109 leader_election.cc:304] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:00:51.539695 29792 raft_consensus.cc:2749] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:00:51.671339 30275 raft_consensus.cc:493] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 02:00:51.671630 30275 raft_consensus.cc:515] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:00:51.672696 30275 leader_election.cc:290] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 02:00:51.674026 23889 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
W20260810 02:00:51.674340 23892 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
I20260810 02:00:51.674485 23892 leader_election.cc:304] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:00:51.674803 30275 raft_consensus.cc:2749] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 02:00:52.311758 29792 raft_consensus.cc:493] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:00:52.311970 29792 raft_consensus.cc:515] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:00:52.312637 29792 leader_election.cc:290] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:00:52.313431 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
W20260810 02:00:52.313627 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
I20260810 02:00:52.313730 24109 leader_election.cc:304] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:00:52.313967 29792 raft_consensus.cc:2749] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:00:52.341081 30300 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 02:00:52.341382 30300 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 02:00:52.342355 30300 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 02:00:52.344625 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 02:00:52.344877 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 02:00:52.344981 26771 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 02:00:52.345276 30300 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260810 02:00:52.428342 23889 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Couldn't send request to peer 50d44445234b4931aaa28141cdaa67ab. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 96: this message will repeat every 5th retry.
W20260810 02:00:53.136132 23892 consensus_peers.cc:597] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9b595b2ec2ff4503a20c1e9e7a7467b7. This is attempt 86: this message will repeat every 5th retry.
I20260810 02:00:54.411012 29792 raft_consensus.cc:493] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:00:54.411274 29792 raft_consensus.cc:515] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:00:54.411840 29792 leader_election.cc:290] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:00:54.413080 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
W20260810 02:00:54.413435 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
I20260810 02:00:54.413583 24109 leader_election.cc:304] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:00:54.413897 29792 raft_consensus.cc:2749] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:00:54.919310 29792 raft_consensus.cc:493] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:00:54.919546 29792 raft_consensus.cc:515] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:00:54.920337 29792 leader_election.cc:290] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:00:54.921427 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
W20260810 02:00:54.921693 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
I20260810 02:00:54.921810 24109 leader_election.cc:304] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:00:54.922040 29792 raft_consensus.cc:2749] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:00:55.195508 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 02:00:55.195758 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 02:00:57.348347 30405 raft_consensus.cc:493] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:00:57.348610 30405 raft_consensus.cc:515] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:00:57.349464 30405 leader_election.cc:290] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:00:57.350464 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
W20260810 02:00:57.350711 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
I20260810 02:00:57.350800 24109 leader_election.cc:304] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:00:57.351055 30405 raft_consensus.cc:2749] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:01:00.198942 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 02:01:00.199200 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 02:01:00.666976 23684 consensus_peers.cc:597] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f88158dec2a449c2a2011b8fc042f0ac. This is attempt 91: this message will repeat every 5th retry.
W20260810 02:01:02.502823 23892 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 101: this message will repeat every 5th retry.
I20260810 02:01:02.704520 30463 raft_consensus.cc:493] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 02:01:02.704764 30463 raft_consensus.cc:515] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 02:01:02.705659 30463 leader_election.cc:290] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 02:01:02.706607 23889 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
W20260810 02:01:02.706774 23892 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
I20260810 02:01:02.706902 23892 leader_election.cc:304] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:01:02.707218 30463 raft_consensus.cc:2749] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 02:01:02.868949 23889 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Couldn't send request to peer 50d44445234b4931aaa28141cdaa67ab. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 101: this message will repeat every 5th retry.
W20260810 02:01:03.160653 23892 consensus_peers.cc:597] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9b595b2ec2ff4503a20c1e9e7a7467b7. This is attempt 91: this message will repeat every 5th retry.
I20260810 02:01:03.385504 30506 raft_consensus.cc:493] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:01:03.385694 30506 raft_consensus.cc:515] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:01:03.386551 30506 leader_election.cc:290] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:01:03.387715 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
W20260810 02:01:03.387959 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
I20260810 02:01:03.388093 24109 leader_election.cc:304] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:01:03.388365 30506 raft_consensus.cc:2749] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:01:04.362484 30506 raft_consensus.cc:493] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:01:04.362735 30506 raft_consensus.cc:515] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:01:04.363595 30506 leader_election.cc:290] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:01:04.364616 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
W20260810 02:01:04.364880 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
I20260810 02:01:04.365020 24109 leader_election.cc:304] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:01:04.365350 30506 raft_consensus.cc:2749] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:01:05.201634 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 02:01:05.201829 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 02:01:05.960917 30565 raft_consensus.cc:493] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 02:01:05.961143 30565 raft_consensus.cc:515] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:01:05.961972 30565 leader_election.cc:290] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 02:01:05.962973 23889 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
W20260810 02:01:05.963192 23892 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
I20260810 02:01:05.963375 23892 leader_election.cc:304] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:01:05.963649 30565 raft_consensus.cc:2749] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 02:01:07.467407 30585 raft_consensus.cc:493] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 02:01:07.467653 30585 raft_consensus.cc:515] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 02:01:07.468546 30585 leader_election.cc:290] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 02:01:07.469558 23892 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
W20260810 02:01:07.469748 23889 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
I20260810 02:01:07.469869 23889 leader_election.cc:304] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:01:07.470101 30585 raft_consensus.cc:2749] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 02:01:08.957849 30572 raft_consensus.cc:493] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:01:08.958089 30572 raft_consensus.cc:515] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:01:08.958847 30572 leader_election.cc:290] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:01:08.959923 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
W20260810 02:01:08.960116 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
I20260810 02:01:08.960216 24109 leader_election.cc:304] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:01:08.960495 30572 raft_consensus.cc:2749] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:01:10.212293 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 02:01:10.212551 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 02:01:10.544456 30647 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 02:01:10.544703 30647 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 02:01:10.545814 30647 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 02:01:10.546638 26771 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260810 02:01:10.548367 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 02:01:10.548537 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 02:01:10.548610 26771 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 02:01:10.548846 30647 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260810 02:01:10.691427 23684 consensus_peers.cc:597] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f88158dec2a449c2a2011b8fc042f0ac. This is attempt 96: this message will repeat every 5th retry.
I20260810 02:01:11.992420 30572 raft_consensus.cc:493] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:01:11.992623 30572 raft_consensus.cc:515] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:01:11.993258 30572 leader_election.cc:290] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:01:11.994064 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
W20260810 02:01:11.994271 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
I20260810 02:01:11.994349 24109 leader_election.cc:304] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:01:11.994606 30572 raft_consensus.cc:493] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:01:11.994814 30572 raft_consensus.cc:515] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:01:11.995030 30684 raft_consensus.cc:2749] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:01:11.995496 30572 leader_election.cc:290] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:01:11.996189 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
W20260810 02:01:11.996397 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
I20260810 02:01:11.996495 24109 leader_election.cc:304] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:01:11.996752 30572 raft_consensus.cc:2749] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 02:01:12.443441 23889 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Couldn't send request to peer 50d44445234b4931aaa28141cdaa67ab. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 106: this message will repeat every 5th retry.
W20260810 02:01:12.456210 23892 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 106: this message will repeat every 5th retry.
I20260810 02:01:12.858850 30699 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 02:01:12.859153 30699 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 02:01:12.859835 30699 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 02:01:12.865272 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 02:01:12.865535 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 02:01:12.865676 26771 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 02:01:12.865952 30699 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260810 02:01:13.423916 30572 raft_consensus.cc:493] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:01:13.424228 30572 raft_consensus.cc:515] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:01:13.424782 30572 leader_election.cc:290] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:01:13.425827 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
W20260810 02:01:13.426085 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
I20260810 02:01:13.426203 24109 leader_election.cc:304] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:01:13.426506 30572 raft_consensus.cc:2749] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:01:13.447491 30572 raft_consensus.cc:493] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:01:13.447716 30572 raft_consensus.cc:515] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
W20260810 02:01:13.449671 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
I20260810 02:01:13.449919 30572 leader_election.cc:290] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:01:13.450191 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
I20260810 02:01:13.450580 24109 leader_election.cc:304] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:01:13.450965 30572 raft_consensus.cc:2749] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 02:01:13.644287 23892 consensus_peers.cc:597] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9b595b2ec2ff4503a20c1e9e7a7467b7. This is attempt 96: this message will repeat every 5th retry.
I20260810 02:01:15.188552 30735 raft_consensus.cc:493] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 02:01:15.188796 30735 raft_consensus.cc:515] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:01:15.189559 30735 leader_election.cc:290] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 02:01:15.190778 23892 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
W20260810 02:01:15.190992 23889 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
I20260810 02:01:15.191128 23889 leader_election.cc:304] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:01:15.191388 30735 raft_consensus.cc:2749] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 02:01:15.215544 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 02:01:15.215732 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 02:01:19.465189 30829 raft_consensus.cc:493] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:01:19.465444 30829 raft_consensus.cc:515] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:01:19.466388 30829 leader_election.cc:290] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:01:19.467592 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
W20260810 02:01:19.467864 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
I20260810 02:01:19.467999 24109 leader_election.cc:304] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:01:19.468292 30829 raft_consensus.cc:2749] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:01:20.218827 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 02:01:20.219081 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 02:01:20.748890 23684 consensus_peers.cc:597] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f88158dec2a449c2a2011b8fc042f0ac. This is attempt 101: this message will repeat every 5th retry.
I20260810 02:01:21.449922 30881 raft_consensus.cc:493] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:01:21.450169 30881 raft_consensus.cc:515] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:01:21.450857 30881 leader_election.cc:290] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:01:21.452054 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
W20260810 02:01:21.452302 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
I20260810 02:01:21.452448 24109 leader_election.cc:304] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:01:21.452749 30881 raft_consensus.cc:2749] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:01:21.880823 30881 raft_consensus.cc:493] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:01:21.881078 30881 raft_consensus.cc:515] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:01:21.881750 30881 leader_election.cc:290] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:01:21.882668 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
W20260810 02:01:21.882921 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
I20260810 02:01:21.883033 24109 leader_election.cc:304] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:01:21.883385 30881 raft_consensus.cc:2749] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 02:01:22.241746 23889 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Couldn't send request to peer 50d44445234b4931aaa28141cdaa67ab. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 111: this message will repeat every 5th retry.
W20260810 02:01:22.863879 23892 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 111: this message will repeat every 5th retry.
W20260810 02:01:23.538902 23892 consensus_peers.cc:597] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9b595b2ec2ff4503a20c1e9e7a7467b7. This is attempt 101: this message will repeat every 5th retry.
I20260810 02:01:23.939956 30943 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 02:01:23.940213 30943 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 02:01:23.940948 30943 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
I20260810 02:01:23.942169 30922 raft_consensus.cc:493] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260810 02:01:23.942348 26771 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111) [suppressed 3 similar messages]
I20260810 02:01:23.942369 30922 raft_consensus.cc:515] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:01:23.943136 30922 leader_election.cc:290] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:01:23.944139 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 02:01:23.944288 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
W20260810 02:01:23.944378 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
W20260810 02:01:23.944545 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
I20260810 02:01:23.944583 26771 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 02:01:23.944687 24109 leader_election.cc:304] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:01:23.944877 30943 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260810 02:01:23.945120 30922 raft_consensus.cc:2749] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:01:24.761718 30873 raft_consensus.cc:493] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 02:01:24.761976 30873 raft_consensus.cc:515] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 02:01:24.762981 30873 leader_election.cc:290] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 02:01:24.763931 23889 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
W20260810 02:01:24.764139 23892 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
I20260810 02:01:24.764267 23892 leader_election.cc:304] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:01:24.764509 30873 raft_consensus.cc:2749] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 02:01:25.059265 30960 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 02:01:25.059459 30960 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 02:01:25.060127 30960 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 02:01:25.062574 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 02:01:25.062834 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 02:01:25.062987 26771 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 02:01:25.063321 30960 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260810 02:01:25.221823 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 02:01:25.222065 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 02:01:27.907982 30993 raft_consensus.cc:493] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 02:01:27.908185 30993 raft_consensus.cc:515] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:01:27.909030 30993 leader_election.cc:290] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 02:01:27.910158 23889 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
W20260810 02:01:27.910342 23892 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
I20260810 02:01:27.910534 23892 leader_election.cc:304] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:01:27.910868 30993 raft_consensus.cc:2749] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 02:01:28.079198 31000 raft_consensus.cc:493] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:01:28.079385 31000 raft_consensus.cc:515] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:01:28.079980 31000 leader_election.cc:290] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:01:28.080987 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
W20260810 02:01:28.081254 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
I20260810 02:01:28.081393 24109 leader_election.cc:304] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:01:28.081712 31000 raft_consensus.cc:2749] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:01:28.205751 30993 raft_consensus.cc:493] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 02:01:28.206053 30993 raft_consensus.cc:515] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 02:01:28.206907 30993 leader_election.cc:290] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 02:01:28.207861 23889 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
W20260810 02:01:28.208053 23892 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
I20260810 02:01:28.208200 23892 leader_election.cc:304] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:01:28.208500 30993 raft_consensus.cc:2749] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 02:01:29.568715 31000 raft_consensus.cc:493] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:01:29.568985 31000 raft_consensus.cc:515] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:01:29.569752 31000 leader_election.cc:290] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:01:29.570936 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
W20260810 02:01:29.571264 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
I20260810 02:01:29.571429 24109 leader_election.cc:304] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:01:29.571727 31000 raft_consensus.cc:2749] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:01:30.225411 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 02:01:30.225649 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 02:01:30.258060 23684 consensus_peers.cc:597] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f88158dec2a449c2a2011b8fc042f0ac. This is attempt 106: this message will repeat every 5th retry.
W20260810 02:01:30.468719 24301 heartbeater.cc:646] Failed to heartbeat to 127.22.69.61:41945 (420 consecutive failures): Network error: Failed to ping master at 127.22.69.61:41945: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 02:01:30.760360 23841 heartbeater.cc:646] Failed to heartbeat to 127.22.69.60:36917 (565 consecutive failures): Network error: Failed to ping master at 127.22.69.60:36917: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
W20260810 02:01:30.927502 24069 heartbeater.cc:646] Failed to heartbeat to 127.22.69.61:41945 (799 consecutive failures): Network error: Failed to ping master at 127.22.69.61:41945: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
I20260810 02:01:31.406744 31000 raft_consensus.cc:493] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:01:31.406945 31000 raft_consensus.cc:515] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:01:31.407585 31000 leader_election.cc:290] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:01:31.408437 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
W20260810 02:01:31.408627 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
I20260810 02:01:31.408700 24109 leader_election.cc:304] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:01:31.408923 31000 raft_consensus.cc:2749] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 02:01:32.244727 23889 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Couldn't send request to peer 50d44445234b4931aaa28141cdaa67ab. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 116: this message will repeat every 5th retry.
W20260810 02:01:32.807916 23892 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 116: this message will repeat every 5th retry.
W20260810 02:01:33.323390 23892 consensus_peers.cc:597] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9b595b2ec2ff4503a20c1e9e7a7467b7. This is attempt 106: this message will repeat every 5th retry.
I20260810 02:01:33.479277 31000 raft_consensus.cc:493] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:01:33.479559 31000 raft_consensus.cc:515] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:01:33.480268 31000 leader_election.cc:290] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:01:33.481351 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
W20260810 02:01:33.481647 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
I20260810 02:01:33.481757 24109 leader_election.cc:304] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:01:33.482116 31000 raft_consensus.cc:2749] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:01:33.749171 31092 raft_consensus.cc:493] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 02:01:33.749394 31092 raft_consensus.cc:515] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:01:33.750156 31092 leader_election.cc:290] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 02:01:33.751283 23892 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
W20260810 02:01:33.751456 23889 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
I20260810 02:01:33.751586 23889 leader_election.cc:304] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:01:33.751827 31092 raft_consensus.cc:2749] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 02:01:35.228925 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 02:01:35.229173 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 02:01:35.495738 31138 raft_consensus.cc:493] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:01:35.495988 31138 raft_consensus.cc:515] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:01:35.496887 31138 leader_election.cc:290] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:01:35.497939 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
W20260810 02:01:35.498214 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
I20260810 02:01:35.498323 24109 leader_election.cc:304] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:01:35.498615 31138 raft_consensus.cc:2749] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 02:01:39.715361 23684 consensus_peers.cc:597] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f88158dec2a449c2a2011b8fc042f0ac. This is attempt 111: this message will repeat every 5th retry.
I20260810 02:01:40.231997 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 02:01:40.232210 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 02:01:41.103191 31238 raft_consensus.cc:493] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:01:41.103478 31238 raft_consensus.cc:515] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:01:41.104223 31238 leader_election.cc:290] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:01:41.105341 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
W20260810 02:01:41.105604 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
I20260810 02:01:41.105759 24109 leader_election.cc:304] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:01:41.106060 31238 raft_consensus.cc:2749] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 02:01:41.742007 23892 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 121: this message will repeat every 5th retry.
I20260810 02:01:41.873024 31251 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 02:01:41.873313 31251 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 02:01:41.874400 31251 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 02:01:41.875991 26771 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260810 02:01:41.877534 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 02:01:41.877708 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 02:01:41.877782 26771 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 02:01:41.878101 31251 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260810 02:01:42.115991 23892 consensus_peers.cc:597] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9b595b2ec2ff4503a20c1e9e7a7467b7. This is attempt 111: this message will repeat every 5th retry.
W20260810 02:01:42.511849 23889 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Couldn't send request to peer 50d44445234b4931aaa28141cdaa67ab. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 121: this message will repeat every 5th retry.
I20260810 02:01:44.476151 31284 raft_consensus.cc:493] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 02:01:44.476365 31284 raft_consensus.cc:515] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:01:44.477322 31284 leader_election.cc:290] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 02:01:44.478428 23892 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
W20260810 02:01:44.478662 23889 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
I20260810 02:01:44.478782 23889 leader_election.cc:304] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:01:44.479005 31284 raft_consensus.cc:2749] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 02:01:45.235445 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 02:01:45.235688 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 02:01:45.355581 31307 raft_consensus.cc:493] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 02:01:45.355783 31307 raft_consensus.cc:515] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 02:01:45.356446 31307 leader_election.cc:290] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 02:01:45.357510 23889 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
W20260810 02:01:45.357704 23892 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
I20260810 02:01:45.357828 23892 leader_election.cc:304] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:01:45.358110 31307 raft_consensus.cc:2749] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 02:01:45.896703 31329 raft_consensus.cc:493] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:01:45.896898 31329 raft_consensus.cc:515] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:01:45.897516 31329 leader_election.cc:290] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:01:45.898406 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
W20260810 02:01:45.898670 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
I20260810 02:01:45.898787 24109 leader_election.cc:304] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:01:45.899096 31329 raft_consensus.cc:2749] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:01:48.259079 31329 raft_consensus.cc:493] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:01:48.259279 31329 raft_consensus.cc:515] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:01:48.260021 31329 leader_election.cc:290] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:01:48.261086 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
W20260810 02:01:48.261296 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
I20260810 02:01:48.261410 24109 leader_election.cc:304] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:01:48.261680 31329 raft_consensus.cc:2749] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:01:48.747478 31329 raft_consensus.cc:493] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:01:48.747676 31329 raft_consensus.cc:515] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:01:48.748344 31329 leader_election.cc:290] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:01:48.749328 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
W20260810 02:01:48.749531 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
I20260810 02:01:48.749608 24109 leader_election.cc:304] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:01:48.749855 31329 raft_consensus.cc:2749] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:01:48.994907 31329 raft_consensus.cc:493] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:01:48.995146 31329 raft_consensus.cc:515] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:01:48.995879 31329 leader_election.cc:290] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:01:48.996840 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
W20260810 02:01:48.997102 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
I20260810 02:01:48.997238 24109 leader_election.cc:304] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:01:48.997611 31329 raft_consensus.cc:2749] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 02:01:49.465302 23684 consensus_peers.cc:597] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f88158dec2a449c2a2011b8fc042f0ac. This is attempt 116: this message will repeat every 5th retry.
I20260810 02:01:50.238692 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 02:01:50.238929 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 02:01:51.910562 31415 raft_consensus.cc:493] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:01:51.910815 31415 raft_consensus.cc:515] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:01:51.911790 31415 leader_election.cc:290] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:01:51.913012 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
W20260810 02:01:51.913287 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
I20260810 02:01:51.913401 24109 leader_election.cc:304] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:01:51.913722 31415 raft_consensus.cc:2749] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 02:01:52.471745 23889 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Couldn't send request to peer 50d44445234b4931aaa28141cdaa67ab. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 126: this message will repeat every 5th retry.
W20260810 02:01:52.492726 23892 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 126: this message will repeat every 5th retry.
W20260810 02:01:52.517926 23892 consensus_peers.cc:597] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9b595b2ec2ff4503a20c1e9e7a7467b7. This is attempt 116: this message will repeat every 5th retry.
I20260810 02:01:54.181879 31455 raft_consensus.cc:493] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 02:01:54.182116 31455 raft_consensus.cc:515] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 02:01:54.182931 31455 leader_election.cc:290] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 02:01:54.183923 23889 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
W20260810 02:01:54.184151 23892 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
I20260810 02:01:54.184324 23892 leader_election.cc:304] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:01:54.184631 31455 raft_consensus.cc:2749] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 02:01:54.435688 31455 raft_consensus.cc:493] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 02:01:54.435959 31455 raft_consensus.cc:515] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:01:54.437009 31455 leader_election.cc:290] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 02:01:54.437994 23889 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
W20260810 02:01:54.438187 23892 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
I20260810 02:01:54.438359 23892 leader_election.cc:304] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:01:54.438673 31455 raft_consensus.cc:2749] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 02:01:54.990257 31446 raft_consensus.cc:493] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:01:54.990428 31446 raft_consensus.cc:515] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:01:54.991065 31446 leader_election.cc:290] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:01:54.992087 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
W20260810 02:01:54.992309 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
I20260810 02:01:54.992381 24109 leader_election.cc:304] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:01:54.992650 31446 raft_consensus.cc:2749] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:01:55.242007 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 02:01:55.242230 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 02:01:59.871313 23684 consensus_peers.cc:597] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f88158dec2a449c2a2011b8fc042f0ac. This is attempt 121: this message will repeat every 5th retry.
I20260810 02:02:00.245155 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 02:02:00.245345 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 02:02:00.557843 31613 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 02:02:00.558064 31613 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 02:02:00.559087 31613 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 02:02:00.559875 26771 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260810 02:02:00.561563 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 02:02:00.561791 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 02:02:00.561915 26771 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 02:02:00.562219 31613 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260810 02:02:01.141889 31576 raft_consensus.cc:493] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:02:01.142100 31576 raft_consensus.cc:515] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:02:01.142887 31576 leader_election.cc:290] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:02:01.144043 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
W20260810 02:02:01.144200 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
I20260810 02:02:01.144255 24109 leader_election.cc:304] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:02:01.144474 31576 raft_consensus.cc:2749] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:02:01.586305 31634 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 02:02:01.586508 31634 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 02:02:01.587162 31634 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 02:02:01.592303 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 02:02:01.592566 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 02:02:01.592703 26771 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 02:02:01.592972 31634 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260810 02:02:02.145726 23892 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 131: this message will repeat every 5th retry.
W20260810 02:02:02.453114 23889 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Couldn't send request to peer 50d44445234b4931aaa28141cdaa67ab. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 131: this message will repeat every 5th retry.
W20260810 02:02:02.892202 23892 consensus_peers.cc:597] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9b595b2ec2ff4503a20c1e9e7a7467b7. This is attempt 121: this message will repeat every 5th retry.
I20260810 02:02:03.399892 31605 raft_consensus.cc:493] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 02:02:03.400095 31605 raft_consensus.cc:515] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:02:03.400964 31605 leader_election.cc:290] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 02:02:03.401993 23889 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
W20260810 02:02:03.402153 23892 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
I20260810 02:02:03.402256 23892 leader_election.cc:304] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:02:03.402469 31605 raft_consensus.cc:2749] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 02:02:03.980120 31678 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 02:02:03.980378 31678 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 02:02:03.981292 31678 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 02:02:03.984318 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 02:02:03.984558 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 02:02:03.984709 26771 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 02:02:03.985018 31678 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260810 02:02:04.350853 31576 raft_consensus.cc:493] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:02:04.351042 31576 raft_consensus.cc:515] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:02:04.351683 31576 leader_election.cc:290] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:02:04.352722 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
W20260810 02:02:04.352944 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
I20260810 02:02:04.353053 24109 leader_election.cc:304] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:02:04.353307 31576 raft_consensus.cc:2749] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:02:04.560248 31576 raft_consensus.cc:493] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:02:04.560441 31576 raft_consensus.cc:515] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:02:04.561056 31576 leader_election.cc:290] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:02:04.562124 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
W20260810 02:02:04.562340 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
I20260810 02:02:04.562418 24109 leader_election.cc:304] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:02:04.562697 31576 raft_consensus.cc:2749] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:02:04.927474 31673 raft_consensus.cc:493] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 02:02:04.927676 31673 raft_consensus.cc:515] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 02:02:04.928369 31673 leader_election.cc:290] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 02:02:04.929414 23892 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
W20260810 02:02:04.929582 23889 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
I20260810 02:02:04.929687 23889 leader_election.cc:304] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:02:04.929883 31673 raft_consensus.cc:2749] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 02:02:05.015722 31576 raft_consensus.cc:493] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:02:05.015911 31576 raft_consensus.cc:515] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:02:05.016639 31576 leader_election.cc:290] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:02:05.017709 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
W20260810 02:02:05.017946 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
I20260810 02:02:05.018016 24109 leader_election.cc:304] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:02:05.018219 31576 raft_consensus.cc:2749] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:02:05.248594 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 02:02:05.248799 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 02:02:07.593685 31576 raft_consensus.cc:493] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:02:07.593905 31576 raft_consensus.cc:515] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:02:07.594794 31576 leader_election.cc:290] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:02:07.595880 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
W20260810 02:02:07.596127 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
I20260810 02:02:07.596215 24109 leader_election.cc:304] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:02:07.596462 31576 raft_consensus.cc:2749] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:02:07.883966 31576 raft_consensus.cc:493] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:02:07.884199 31576 raft_consensus.cc:515] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:02:07.884795 31576 leader_election.cc:290] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:02:07.885849 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
W20260810 02:02:07.886101 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
I20260810 02:02:07.886178 24109 leader_election.cc:304] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:02:07.886476 31576 raft_consensus.cc:2749] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:02:08.483376 31765 raft_consensus.cc:493] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 02:02:08.483546 31765 raft_consensus.cc:515] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 02:02:08.484263 31765 leader_election.cc:290] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 02:02:08.485330 23889 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
W20260810 02:02:08.486169 23892 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
I20260810 02:02:08.486284 23892 leader_election.cc:304] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:02:08.487248 31765 raft_consensus.cc:2749] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 02:02:09.612747 23684 consensus_peers.cc:597] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f88158dec2a449c2a2011b8fc042f0ac. This is attempt 126: this message will repeat every 5th retry.
I20260810 02:02:10.251963 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 02:02:10.252161 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 02:02:11.746886 23892 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 136: this message will repeat every 5th retry.
I20260810 02:02:11.928156 31859 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 02:02:11.928403 31859 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 02:02:11.929278 31859 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 02:02:11.930720 26771 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111) [suppressed 5 similar messages]
W20260810 02:02:11.932432 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 02:02:11.932821 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 02:02:11.932968 26771 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 02:02:11.933251 31859 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260810 02:02:12.219270 23889 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Couldn't send request to peer 50d44445234b4931aaa28141cdaa67ab. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 136: this message will repeat every 5th retry.
W20260810 02:02:12.382925 23892 consensus_peers.cc:597] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9b595b2ec2ff4503a20c1e9e7a7467b7. This is attempt 126: this message will repeat every 5th retry.
I20260810 02:02:13.886662 31576 raft_consensus.cc:493] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:02:13.886850 31576 raft_consensus.cc:515] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:02:13.887768 31576 leader_election.cc:290] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:02:13.889097 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
W20260810 02:02:13.889317 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
I20260810 02:02:13.889415 24109 leader_election.cc:304] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:02:13.889679 31576 raft_consensus.cc:2749] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:02:14.121359 31905 raft_consensus.cc:493] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 02:02:14.121590 31905 raft_consensus.cc:515] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 02:02:14.122260 31905 leader_election.cc:290] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 02:02:14.123195 23889 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
W20260810 02:02:14.123315 23892 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
I20260810 02:02:14.123461 23892 leader_election.cc:304] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:02:14.123701 31905 raft_consensus.cc:2749] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 02:02:15.255278 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 02:02:15.255520 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 02:02:16.826416 31576 raft_consensus.cc:493] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:02:16.826597 31576 raft_consensus.cc:515] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:02:16.827409 31576 leader_election.cc:290] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:02:16.828390 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
W20260810 02:02:16.828735 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
I20260810 02:02:16.828902 24109 leader_election.cc:304] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:02:16.829166 31576 raft_consensus.cc:2749] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:02:17.204802 31935 raft_consensus.cc:493] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 02:02:17.204996 31935 raft_consensus.cc:515] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:02:17.205651 31935 leader_election.cc:290] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 02:02:17.206576 23889 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
W20260810 02:02:17.206724 23892 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
I20260810 02:02:17.206841 23892 leader_election.cc:304] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:02:17.207088 31935 raft_consensus.cc:2749] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 02:02:19.698742 31991 raft_consensus.cc:493] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:02:19.698937 31991 raft_consensus.cc:515] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:02:19.699708 31991 leader_election.cc:290] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:02:19.700789 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
W20260810 02:02:19.701012 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
I20260810 02:02:19.701102 24109 leader_election.cc:304] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:02:19.701402 31991 raft_consensus.cc:2749] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 02:02:19.961059 23684 consensus_peers.cc:597] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f88158dec2a449c2a2011b8fc042f0ac. This is attempt 131: this message will repeat every 5th retry.
I20260810 02:02:20.258670 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 02:02:20.258929 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 02:02:21.935593 23892 consensus_peers.cc:597] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9b595b2ec2ff4503a20c1e9e7a7467b7. This is attempt 131: this message will repeat every 5th retry.
W20260810 02:02:22.141705 23892 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 141: this message will repeat every 5th retry.
W20260810 02:02:22.150539 23889 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Couldn't send request to peer 50d44445234b4931aaa28141cdaa67ab. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 141: this message will repeat every 5th retry.
I20260810 02:02:22.187594 32052 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 02:02:22.187939 32052 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 02:02:22.188828 32052 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 02:02:22.190290 26771 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260810 02:02:22.191856 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 02:02:22.192080 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 02:02:22.192175 26771 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 02:02:22.192418 32052 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260810 02:02:23.937736 31991 raft_consensus.cc:493] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:02:23.938001 31991 raft_consensus.cc:515] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:02:23.939136 31991 leader_election.cc:290] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:02:23.940042 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
W20260810 02:02:23.940302 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
I20260810 02:02:23.940418 24109 leader_election.cc:304] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:02:23.940675 31991 raft_consensus.cc:2749] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:02:24.536778 31991 raft_consensus.cc:493] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:02:24.537015 31991 raft_consensus.cc:515] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:02:24.537781 31991 leader_election.cc:290] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:02:24.538874 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
W20260810 02:02:24.539114 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
I20260810 02:02:24.539188 24109 leader_election.cc:304] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:02:24.539471 31991 raft_consensus.cc:2749] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:02:25.262339 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 02:02:25.262600 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 02:02:25.604099 32101 raft_consensus.cc:493] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 02:02:25.604374 32101 raft_consensus.cc:515] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:02:25.605571 32101 leader_election.cc:290] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 02:02:25.606649 23889 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
W20260810 02:02:25.606817 23892 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
I20260810 02:02:25.606987 23892 leader_election.cc:304] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:02:25.607308 32101 raft_consensus.cc:2749] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 02:02:25.678474 31991 raft_consensus.cc:493] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:02:25.678673 31991 raft_consensus.cc:515] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:02:25.679356 31991 leader_election.cc:290] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:02:25.680409 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
W20260810 02:02:25.680689 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
I20260810 02:02:25.680796 24109 leader_election.cc:304] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:02:25.681100 31991 raft_consensus.cc:2749] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:02:27.547029 32169 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 02:02:27.547389 32169 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 02:02:27.548377 32169 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 02:02:27.549744 26771 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260810 02:02:27.552615 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 02:02:27.552850 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 02:02:27.552940 26771 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 02:02:27.553218 32169 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260810 02:02:29.407006 23684 consensus_peers.cc:597] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f88158dec2a449c2a2011b8fc042f0ac. This is attempt 136: this message will repeat every 5th retry.
I20260810 02:02:30.266222 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 02:02:30.266608 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 02:02:30.622535 24301 heartbeater.cc:646] Failed to heartbeat to 127.22.69.61:41945 (540 consecutive failures): Network error: Failed to ping master at 127.22.69.61:41945: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 02:02:30.818956 23841 heartbeater.cc:646] Failed to heartbeat to 127.22.69.60:36917 (716 consecutive failures): Network error: Failed to ping master at 127.22.69.60:36917: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
W20260810 02:02:31.070524 24063 heartbeater.cc:646] Failed to heartbeat to 127.22.69.60:36917 (1072 consecutive failures): Network error: Failed to ping master at 127.22.69.60:36917: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
W20260810 02:02:31.070967 23892 consensus_peers.cc:597] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9b595b2ec2ff4503a20c1e9e7a7467b7. This is attempt 136: this message will repeat every 5th retry.
I20260810 02:02:31.925493 31991 raft_consensus.cc:493] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:02:31.925733 31991 raft_consensus.cc:515] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:02:31.926524 31991 leader_election.cc:290] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:02:31.927637 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
W20260810 02:02:31.927886 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
I20260810 02:02:31.927991 24109 leader_election.cc:304] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:02:31.928282 31991 raft_consensus.cc:2749] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 02:02:32.117726 23889 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Couldn't send request to peer 50d44445234b4931aaa28141cdaa67ab. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 146: this message will repeat every 5th retry.
W20260810 02:02:32.685421 23892 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 146: this message will repeat every 5th retry.
I20260810 02:02:33.351115 32283 raft_consensus.cc:493] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 02:02:33.351415 32283 raft_consensus.cc:515] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 02:02:33.352058 32283 leader_election.cc:290] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 02:02:33.352996 23889 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
W20260810 02:02:33.353129 23892 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
I20260810 02:02:33.353281 23892 leader_election.cc:304] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:02:33.353542 32283 raft_consensus.cc:2749] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 02:02:35.269425 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 02:02:35.269650 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 02:02:35.541853 32305 raft_consensus.cc:493] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:02:35.542104 32305 raft_consensus.cc:515] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:02:35.542861 32305 leader_election.cc:290] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:02:35.543896 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
W20260810 02:02:35.544130 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
I20260810 02:02:35.544221 24109 leader_election.cc:304] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:02:35.544507 32305 raft_consensus.cc:2749] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:02:35.813827 32340 raft_consensus.cc:493] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 02:02:35.814066 32340 raft_consensus.cc:515] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:02:35.814841 32340 leader_election.cc:290] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 02:02:35.815912 23889 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
W20260810 02:02:35.816036 23892 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
I20260810 02:02:35.816180 23892 leader_election.cc:304] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:02:35.816466 32340 raft_consensus.cc:2749] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 02:02:36.024552 32348 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 02:02:36.024752 32348 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 02:02:36.025553 32348 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 02:02:36.027273 26771 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260810 02:02:36.032567 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 02:02:36.032862 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 02:02:36.033026 26771 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 02:02:36.033360 32348 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260810 02:02:36.247829 32305 raft_consensus.cc:493] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:02:36.248003 32305 raft_consensus.cc:515] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:02:36.248805 32305 leader_election.cc:290] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:02:36.249640 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
W20260810 02:02:36.249884 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
I20260810 02:02:36.249977 24109 leader_election.cc:304] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:02:36.250176 32305 raft_consensus.cc:2749] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:02:37.992969 32400 raft_consensus.cc:493] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:02:37.993134 32400 raft_consensus.cc:515] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:02:37.993741 32400 leader_election.cc:290] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:02:37.994583 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
W20260810 02:02:37.994752 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
I20260810 02:02:37.994810 24109 leader_election.cc:304] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:02:37.995010 32400 raft_consensus.cc:2749] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 02:02:39.732476 23684 consensus_peers.cc:597] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f88158dec2a449c2a2011b8fc042f0ac. This is attempt 141: this message will repeat every 5th retry.
W20260810 02:02:40.190708 23892 consensus_peers.cc:597] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9b595b2ec2ff4503a20c1e9e7a7467b7. This is attempt 141: this message will repeat every 5th retry.
I20260810 02:02:40.272475 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 02:02:40.272694 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 02:02:40.864249 32450 raft_consensus.cc:493] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 02:02:40.864501 32450 raft_consensus.cc:515] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 02:02:40.865406 32450 leader_election.cc:290] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 02:02:40.866503 23889 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
W20260810 02:02:40.866686 23892 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
I20260810 02:02:40.866782 23892 leader_election.cc:304] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:02:40.866983 32450 raft_consensus.cc:2749] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 02:02:41.163918 32400 raft_consensus.cc:493] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:02:41.164117 32400 raft_consensus.cc:515] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:02:41.164785 32400 leader_election.cc:290] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:02:41.165789 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
W20260810 02:02:41.165969 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
I20260810 02:02:41.166035 24109 leader_election.cc:304] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:02:41.166285 32400 raft_consensus.cc:2749] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 02:02:41.424813 23889 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Couldn't send request to peer 50d44445234b4931aaa28141cdaa67ab. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 151: this message will repeat every 5th retry.
W20260810 02:02:42.292073 23892 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 151: this message will repeat every 5th retry.
I20260810 02:02:43.032835 32493 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 02:02:43.033067 32493 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 02:02:43.033820 32493 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 02:02:43.035145 26771 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260810 02:02:43.040427 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 02:02:43.040709 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 02:02:43.040824 26771 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 02:02:43.041184 32493 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260810 02:02:45.276081 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 02:02:45.276302 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 02:02:49.805160 23684 consensus_peers.cc:597] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f88158dec2a449c2a2011b8fc042f0ac. This is attempt 146: this message will repeat every 5th retry.
W20260810 02:02:49.963487 23892 consensus_peers.cc:597] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9b595b2ec2ff4503a20c1e9e7a7467b7. This is attempt 146: this message will repeat every 5th retry.
I20260810 02:02:50.279994 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 02:02:50.280258 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 02:02:50.828132 32649 raft_consensus.cc:493] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:02:50.828367 32649 raft_consensus.cc:515] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:02:50.829263 32649 leader_election.cc:290] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:02:50.830578 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
W20260810 02:02:50.830811 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
I20260810 02:02:50.830922 24109 leader_election.cc:304] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:02:50.831226 32649 raft_consensus.cc:2749] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:02:51.128036 32664 raft_consensus.cc:493] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 02:02:51.128207 32664 raft_consensus.cc:515] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:02:51.128971 32664 leader_election.cc:290] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 02:02:51.130049 23892 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
W20260810 02:02:51.130271 23889 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
I20260810 02:02:51.130389 23889 leader_election.cc:304] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:02:51.130685 32664 raft_consensus.cc:2749] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 02:02:51.189054 23889 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Couldn't send request to peer 50d44445234b4931aaa28141cdaa67ab. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 156: this message will repeat every 5th retry.
I20260810 02:02:51.525599 32649 raft_consensus.cc:493] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:02:51.525821 32649 raft_consensus.cc:515] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:02:51.526423 32649 leader_election.cc:290] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:02:51.527652 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
W20260810 02:02:51.527872 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
I20260810 02:02:51.527942 24109 leader_election.cc:304] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:02:51.528224 32649 raft_consensus.cc:2749] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 02:02:52.123401 23892 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 156: this message will repeat every 5th retry.
I20260810 02:02:52.937100 32649 raft_consensus.cc:493] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:02:52.937332 32649 raft_consensus.cc:515] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:02:52.937920 32649 leader_election.cc:290] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:02:52.939039 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
W20260810 02:02:52.939332 24109 leader_election.cc:343] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 6ca335f1e5964f8490a49ba4c4fb4c72
I20260810 02:02:52.939476 24109 leader_election.cc:304] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:02:52.939831 32649 raft_consensus.cc:2749] T 6ca335f1e5964f8490a49ba4c4fb4c72 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:02:53.275134 32690 raft_consensus.cc:493] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 02:02:53.275357 32690 raft_consensus.cc:515] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 02:02:53.276218 32690 leader_election.cc:290] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 02:02:53.277429 23892 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
W20260810 02:02:53.277606 23889 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
I20260810 02:02:53.277698 23889 leader_election.cc:304] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:02:53.277894 32690 raft_consensus.cc:2749] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 02:02:54.330138 32728 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 02:02:54.330456 32728 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 02:02:54.331620 32728 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 02:02:54.333163 26771 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260810 02:02:54.335477 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 02:02:54.335767 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 02:02:54.335970 26771 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 02:02:54.336404 32728 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260810 02:02:54.477145 32649 raft_consensus.cc:493] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:02:54.477401 32649 raft_consensus.cc:515] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:02:54.478219 32649 leader_election.cc:290] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:02:54.479568 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
W20260810 02:02:54.479825 24109 leader_election.cc:343] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 0dc1d64a95b84b26b17a29a67b5e3a8d
I20260810 02:02:54.479924 24109 leader_election.cc:304] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:02:54.480208 32649 raft_consensus.cc:2749] T 0dc1d64a95b84b26b17a29a67b5e3a8d P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:02:55.127321 32747 raft_consensus.cc:493] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 02:02:55.127537 32747 raft_consensus.cc:515] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:02:55.128134 32747 leader_election.cc:290] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 02:02:55.129328 23889 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
W20260810 02:02:55.129520 23892 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
I20260810 02:02:55.129631 23892 leader_election.cc:304] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:02:55.129868 32747 raft_consensus.cc:2749] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 02:02:55.283506 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 02:02:55.283710 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 02:02:55.897455 32766 raft_consensus.cc:493] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:02:55.897639 32766 raft_consensus.cc:515] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:02:55.898279 32766 leader_election.cc:290] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:02:55.899600 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
W20260810 02:02:55.899829 24109 leader_election.cc:343] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: cad2404759604bfea7c273e1aa40539a
I20260810 02:02:55.899937 24109 leader_election.cc:304] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:02:55.900231 32766 raft_consensus.cc:2749] T cad2404759604bfea7c273e1aa40539a P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:02:56.377125 311 raft_consensus.cc:493] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1422349180c446db6c1768b3025253c)
I20260810 02:02:56.377363 311 raft_consensus.cc:515] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "919313c0dee540a690f46dbc8208d20b" member_type: VOTER last_known_addr { host: "127.22.69.62" port: 35311 } } peers { permanent_uuid: "b1422349180c446db6c1768b3025253c" member_type: VOTER last_known_addr { host: "127.22.69.61" port: 41945 } } peers { permanent_uuid: "757bb3f336084510845ed58df264c721" member_type: VOTER last_known_addr { host: "127.22.69.60" port: 36917 } }
I20260810 02:02:56.378311 311 leader_election.cc:290] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b1422349180c446db6c1768b3025253c (127.22.69.61:41945), 757bb3f336084510845ed58df264c721 (127.22.69.60:36917)
W20260810 02:02:56.381033 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b1422349180c446db6c1768b3025253c (127.22.69.61:41945): Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111)
W20260810 02:02:56.381260 26771 leader_election.cc:336] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 757bb3f336084510845ed58df264c721 (127.22.69.60:36917): Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
I20260810 02:02:56.381433 26771 leader_election.cc:304] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 919313c0dee540a690f46dbc8208d20b; no voters: 757bb3f336084510845ed58df264c721, b1422349180c446db6c1768b3025253c
I20260810 02:02:56.381719 311 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 919313c0dee540a690f46dbc8208d20b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260810 02:02:57.333621 323 raft_consensus.cc:493] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 02:02:57.333827 323 raft_consensus.cc:515] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } }
I20260810 02:02:57.334621 323 leader_election.cc:290] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 02:02:57.335692 23889 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
W20260810 02:02:57.335824 23892 leader_election.cc:343] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 09ed68395e2d44a5b51a22abe2203406
I20260810 02:02:57.335963 23892 leader_election.cc:304] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:02:57.336246 323 raft_consensus.cc:2749] T 09ed68395e2d44a5b51a22abe2203406 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 02:02:57.613207 32766 raft_consensus.cc:493] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:02:57.613426 32766 raft_consensus.cc:515] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:02:57.614197 32766 leader_election.cc:290] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:02:57.615265 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
W20260810 02:02:57.615437 24109 leader_election.cc:343] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 5c49a207b37e40579d34002c3fb57afb
I20260810 02:02:57.615502 24109 leader_election.cc:304] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:02:57.615705 32766 raft_consensus.cc:2749] T 5c49a207b37e40579d34002c3fb57afb P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 02:02:58.255769 323 raft_consensus.cc:493] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 498d352dbea9405199a121e84a4ab926)
I20260810 02:02:58.256013 323 raft_consensus.cc:515] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:02:58.256780 323 leader_election.cc:290] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597)
W20260810 02:02:58.257807 23889 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
W20260810 02:02:58.257970 23892 leader_election.cc:343] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Not found: Tablet not found: 89d9556cc526444299c786de606bb5b1
I20260810 02:02:58.258128 23892 leader_election.cc:304] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48e33bc1e8e94d4887bb64214d0b6c36; no voters: 498d352dbea9405199a121e84a4ab926, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:02:58.258400 323 raft_consensus.cc:2749] T 89d9556cc526444299c786de606bb5b1 P 48e33bc1e8e94d4887bb64214d0b6c36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 02:02:59.720013 23684 consensus_peers.cc:597] T f88158dec2a449c2a2011b8fc042f0ac P 50d44445234b4931aaa28141cdaa67ab -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f88158dec2a449c2a2011b8fc042f0ac. This is attempt 151: this message will repeat every 5th retry.
W20260810 02:03:00.066447 23892 consensus_peers.cc:597] T 9b595b2ec2ff4503a20c1e9e7a7467b7 P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 498d352dbea9405199a121e84a4ab926 (127.22.69.3:38597): Couldn't send request to peer 498d352dbea9405199a121e84a4ab926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9b595b2ec2ff4503a20c1e9e7a7467b7. This is attempt 151: this message will repeat every 5th retry.
I20260810 02:03:00.287010 26745 master_runner.cc:187] Error getting master registration for 127.22.69.61:41945: Network error: Client connection negotiation failed: client connection to 127.22.69.61:41945: connect: Connection refused (error 111),
I20260810 02:03:00.287292 26745 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 02:03:00.399240 26781 maintenance_manager.cc:419] P 919313c0dee540a690f46dbc8208d20b: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1729 bytes on disk
I20260810 02:03:00.405030 26780 maintenance_manager.cc:643] P 919313c0dee540a690f46dbc8208d20b: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.005s sys 0.000s Metrics: {"dirs.queue_time_us":376,"thread_start_us":550,"threads_started":2,"wal-append.queue_time_us":438}
I20260810 02:03:00.470799 32766 raft_consensus.cc:493] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 02:03:00.471027 32766 raft_consensus.cc:515] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d44445234b4931aaa28141cdaa67ab" member_type: VOTER last_known_addr { host: "127.22.69.1" port: 45491 } } peers { permanent_uuid: "498d352dbea9405199a121e84a4ab926" member_type: VOTER last_known_addr { host: "127.22.69.3" port: 38597 } } peers { permanent_uuid: "48e33bc1e8e94d4887bb64214d0b6c36" member_type: VOTER last_known_addr { host: "127.22.69.2" port: 46271 } }
I20260810 02:03:00.471843 32766 leader_election.cc:290] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491), 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271)
W20260810 02:03:00.472669 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48e33bc1e8e94d4887bb64214d0b6c36 (127.22.69.2:46271): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
W20260810 02:03:00.472893 24109 leader_election.cc:343] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Not found: Tablet not found: d6cdef730e234b67843af88af2d1a6c2
I20260810 02:03:00.473009 24109 leader_election.cc:304] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498d352dbea9405199a121e84a4ab926; no voters: 48e33bc1e8e94d4887bb64214d0b6c36, 50d44445234b4931aaa28141cdaa67ab
I20260810 02:03:00.473258 32766 raft_consensus.cc:2749] T d6cdef730e234b67843af88af2d1a6c2 P 498d352dbea9405199a121e84a4ab926 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 02:03:00.715752 24317 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
W20260810 02:03:00.716086 24317 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
W20260810 02:03:00.716202 24317 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
W20260810 02:03:00.716288 24317 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
F20260810 02:03:00.727149 24366 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260810 02:03:00.727424 24341 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b03802d8f2f14a24b00ab6efbe9392dc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
*** Aborted at 1786327380 (unix time) try "date -d @1786327380" if you are using GNU date ***
*** Check failure stack trace: ***
F20260810 02:03:00.727424 24341 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b03802d8f2f14a24b00ab6efbe9392dc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728493 24378 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260810 02:03:00.727424 24341 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b03802d8f2f14a24b00ab6efbe9392dc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728493 24378 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728570 24375 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260810 02:03:00.727424 24341 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b03802d8f2f14a24b00ab6efbe9392dc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728493 24378 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728570 24375 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728693 24368 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260810 02:03:00.727424 24341 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b03802d8f2f14a24b00ab6efbe9392dc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728493 24378 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728570 24375 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728693 24368 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.729089 24356 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260810 02:03:00.727424 24341 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b03802d8f2f14a24b00ab6efbe9392dc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728493 24378 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728570 24375 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728693 24368 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.729089 24356 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730182 24353 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260810 02:03:00.727424 24341 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b03802d8f2f14a24b00ab6efbe9392dc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728493 24378 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728570 24375 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728693 24368 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.729089 24356 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730182 24353 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730384 24333 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1bd23f5807da48238cbddc7597b8ab50 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260810 02:03:00.727424 24341 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b03802d8f2f14a24b00ab6efbe9392dc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728493 24378 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728570 24375 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728693 24368 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.729089 24356 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730182 24353 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730384 24333 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1bd23f5807da48238cbddc7597b8ab50 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730603 24339 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_73822ba0bac443d19cc7c66f6f547bda on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260810 02:03:00.727424 24341 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b03802d8f2f14a24b00ab6efbe9392dc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728493 24378 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728570 24375 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728693 24368 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.729089 24356 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730182 24353 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730384 24333 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1bd23f5807da48238cbddc7597b8ab50 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730603 24339 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_73822ba0bac443d19cc7c66f6f547bda on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731405 24347 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260810 02:03:00.727424 24341 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b03802d8f2f14a24b00ab6efbe9392dc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728493 24378 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728570 24375 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728693 24368 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.729089 24356 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730182 24353 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730384 24333 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1bd23f5807da48238cbddc7597b8ab50 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730603 24339 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_73822ba0bac443d19cc7c66f6f547bda on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731405 24347 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731812 24337 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5a4f0e1bcc16485ab9a7148d13d21a68 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260810 02:03:00.727424 24341 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b03802d8f2f14a24b00ab6efbe9392dc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728493 24378 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728570 24375 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728693 24368 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.729089 24356 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730182 24353 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730384 24333 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1bd23f5807da48238cbddc7597b8ab50 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730603 24339 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_73822ba0bac443d19cc7c66f6f547bda on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731405 24347 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731812 24337 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5a4f0e1bcc16485ab9a7148d13d21a68 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731979 24342 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_70288f170151490fb6ba63a3bf29f01e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260810 02:03:00.727424 24341 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b03802d8f2f14a24b00ab6efbe9392dc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728493 24378 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728570 24375 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728693 24368 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.729089 24356 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730182 24353 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730384 24333 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1bd23f5807da48238cbddc7597b8ab50 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730603 24339 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_73822ba0bac443d19cc7c66f6f547bda on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731405 24347 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731812 24337 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5a4f0e1bcc16485ab9a7148d13d21a68 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731979 24342 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_70288f170151490fb6ba63a3bf29f01e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.732048 24346 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260810 02:03:00.727424 24341 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b03802d8f2f14a24b00ab6efbe9392dc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728493 24378 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728570 24375 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728693 24368 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.729089 24356 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730182 24353 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730384 24333 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1bd23f5807da48238cbddc7597b8ab50 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730603 24339 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_73822ba0bac443d19cc7c66f6f547bda on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731405 24347 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731812 24337 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5a4f0e1bcc16485ab9a7148d13d21a68 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731979 24342 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_70288f170151490fb6ba63a3bf29f01e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.732048 24346 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.732065 24358 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260810 02:03:00.727424 24341 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b03802d8f2f14a24b00ab6efbe9392dc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728493 24378 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728570 24375 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728693 24368 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.729089 24356 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730182 24353 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730384 24333 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1bd23f5807da48238cbddc7597b8ab50 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730603 24339 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_73822ba0bac443d19cc7c66f6f547bda on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731405 24347 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731812 24337 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5a4f0e1bcc16485ab9a7148d13d21a68 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731979 24342 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_70288f170151490fb6ba63a3bf29f01e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.732048 24346 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.732065 24358 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.733533 24345 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260810 02:03:00.727424 24341 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b03802d8f2f14a24b00ab6efbe9392dc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728493 24378 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728570 24375 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728693 24368 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.729089 24356 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730182 24353 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730384 24333 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1bd23f5807da48238cbddc7597b8ab50 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730603 24339 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_73822ba0bac443d19cc7c66f6f547bda on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731405 24347 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731812 24337 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5a4f0e1bcc16485ab9a7148d13d21a68 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731979 24342 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_70288f170151490fb6ba63a3bf29f01e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.732048 24346 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.732065 24358 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.733533 24345 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.733706 24350 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260810 02:03:00.727424 24341 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b03802d8f2f14a24b00ab6efbe9392dc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728493 24378 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728570 24375 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728693 24368 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.729089 24356 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730182 24353 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730384 24333 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1bd23f5807da48238cbddc7597b8ab50 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730603 24339 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_73822ba0bac443d19cc7c66f6f547bda on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731405 24347 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731812 24337 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5a4f0e1bcc16485ab9a7148d13d21a68 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731979 24342 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_70288f170151490fb6ba63a3bf29f01e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.732048 24346 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.732065 24358 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.733533 24345 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.733706 24350 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.733768 24351 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260810 02:03:00.727424 24341 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b03802d8f2f14a24b00ab6efbe9392dc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728493 24378 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728570 24375 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728693 24368 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.729089 24356 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730182 24353 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730384 24333 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1bd23f5807da48238cbddc7597b8ab50 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730603 24339 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_73822ba0bac443d19cc7c66f6f547bda on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731405 24347 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731812 24337 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5a4f0e1bcc16485ab9a7148d13d21a68 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731979 24342 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_70288f170151490fb6ba63a3bf29f01e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.732048 24346 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.732065 24358 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.733533 24345 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.733706 24350 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.733768 24351 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.733939 24359 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260810 02:03:00.727424 24341 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b03802d8f2f14a24b00ab6efbe9392dc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728493 24378 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728570 24375 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728693 24368 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.729089 24356 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730182 24353 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730384 24333 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1bd23f5807da48238cbddc7597b8ab50 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730603 24339 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_73822ba0bac443d19cc7c66f6f547bda on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731405 24347 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731812 24337 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5a4f0e1bcc16485ab9a7148d13d21a68 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731979 24342 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_70288f170151490fb6ba63a3bf29f01e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.732048 24346 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.732065 24358 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.733533 24345 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.733706 24350 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.733768 24351 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.733939 24359 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.735194 24336 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9175c2bd9b0542f2a6b95eae8a882b7a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260810 02:03:00.727424 24341 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b03802d8f2f14a24b00ab6efbe9392dc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728493 24378 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728570 24375 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728693 24368 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.729089 24356 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730182 24353 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730384 24333 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1bd23f5807da48238cbddc7597b8ab50 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730603 24339 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_73822ba0bac443d19cc7c66f6f547bda on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731405 24347 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731812 24337 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5a4f0e1bcc16485ab9a7148d13d21a68 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731979 24342 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_70288f170151490fb6ba63a3bf29f01e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.732048 24346 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.732065 24358 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.733533 24345 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.733706 24350 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.733768 24351 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.733939 24359 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.735194 24336 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9175c2bd9b0542f2a6b95eae8a882b7a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.735263 24340 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_7e5efb367a02411aac568a75dd39b274 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260810 02:03:00.727424 24341 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b03802d8f2f14a24b00ab6efbe9392dc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728493 24378 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728570 24375 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728693 24368 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.729089 24356 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730182 24353 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730384 24333 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1bd23f5807da48238cbddc7597b8ab50 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730603 24339 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_73822ba0bac443d19cc7c66f6f547bda on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731405 24347 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731812 24337 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5a4f0e1bcc16485ab9a7148d13d21a68 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731979 24342 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_70288f170151490fb6ba63a3bf29f01e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.732048 24346 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.732065 24358 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.733533 24345 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.733706 24350 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.733768 24351 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.733939 24359 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.735194 24336 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9175c2bd9b0542f2a6b95eae8a882b7a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.735263 24340 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_7e5efb367a02411aac568a75dd39b274 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.735813 24338 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_14c5bada100e48adb5f378e73ee7d00b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
*** Check failure stack trace: ***
PC: @ 0x0 (unknown)
F20260810 02:03:00.727424 24341 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b03802d8f2f14a24b00ab6efbe9392dc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728493 24378 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728570 24375 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728693 24368 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.729089 24356 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730182 24353 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730384 24333 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1bd23f5807da48238cbddc7597b8ab50 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730603 24339 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_73822ba0bac443d19cc7c66f6f547bda on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731405 24347 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731812 24337 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5a4f0e1bcc16485ab9a7148d13d21a68 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731979 24342 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_70288f170151490fb6ba63a3bf29f01e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.732048 24346 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.732065 24358 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.733533 24345 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.733706 24350 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.733768 24351 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.733939 24359 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.735194 24336 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9175c2bd9b0542f2a6b95eae8a882b7a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.735263 24340 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_7e5efb367a02411aac568a75dd39b274 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.735813 24338 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_14c5bada100e48adb5f378e73ee7d00b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.736704 24331 master-stress-test.cc:299] Check failed: _s.ok() Bad status: F20260810 02:03:Timed out: Error creating table default.Table_e3c77b251fdf471d8916f06da94be990 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111) 24332 master-stress-test.cc:299] Check failed: _s.ok() Bad status: T*** Check failure stack trace: ***
F20260810 02:03:00.727424 24341 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b03802d8f2f14a24b00ab6efbe9392dc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728493 24378 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728570 24375 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728693 24368 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.729089 24356 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730182 24353 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730384 24333 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1bd23f5807da48238cbddc7597b8ab50 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730603 24339 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_73822ba0bac443d19cc7c66f6f547bda on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731405 24347 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731812 24337 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5a4f0e1bcc16485ab9a7148d13d21a68 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731979 24342 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_70288f170151490fb6ba63a3bf29f01e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.732048 24346 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.732065 24358 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.733533 24345 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.733706 24350 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.733768 24351 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.733939 24359 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.735194 24336 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9175c2bd9b0542f2a6b95eae8a882b7a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.735263 24340 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_7e5efb367a02411aac568a75dd39b274 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.735813 24338 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_14c5bada100e48adb5f378e73ee7d00b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.736704 24331 master-stress-test.cc:299] Check failed: _s.ok() Bad status: F20260810 02:03:Timed out: Error creating table default.Table_e3c77b251fdf471d8916f06da94be990 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111) 24332 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_bb2d4e224bd14be09d60bce7977d4c19 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
*** Check failure stack trace: ***
*** SIGABRT (@0x3e800005914) received by PID 22804 (TID 0x7f19072f6700) from PID 22804; stack trace: ***
F20260810 02:03:00.727424 24341 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b03802d8f2f14a24b00ab6efbe9392dc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728493 24378 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728570 24375 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.728693 24368 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.729089 24356 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730182 24353 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730384 24333 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1bd23f5807da48238cbddc7597b8ab50 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.730603 24339 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_73822ba0bac443d19cc7c66f6f547bda on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731405 24347 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731812 24337 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5a4f0e1bcc16485ab9a7148d13d21a68 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.731979 24342 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_70288f170151490fb6ba63a3bf29f01e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.732048 24346 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.732065 24358 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.733533 24345 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.733706 24350 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.733768 24351 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.733939 24359 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.735194 24336 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9175c2bd9b0542f2a6b95eae8a882b7a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.735263 24340 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_7e5efb367a02411aac568a75dd39b274 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.735813 24338 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_14c5bada100e48adb5f378e73ee7d00b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.736704 24331 master-stress-test.cc:299] Check failed: _s.ok() Bad status: F20260810 02:03:Timed out: Error creating table default.Table_e3c77b251fdf471d8916f06da94be990 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111) 24332 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_bb2d4e224bd14be09d60bce7977d4c19 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)F20260810 02:03:00.736775 24373 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.22.69.62:35311,127.22.69.61:41945,127.22.69.60:36917, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.22.69.60:36917: connect: Connection refused (error 111)
*** Check failure stack trace: ***
@ 0x7f1923f69980 (unknown) at ??:0
@ 0x7f191cd35e87 gsignal at ??:0
@ 0x7f191cd377f1 abort at ??:0
@ 0x7f191fa7dd5d google::LogMessage::Fail() at ??:0
@ 0x7f191fa81b23 google::LogMessage::SendToLog() at ??:0
@ 0x7f191fa7d75c google::LogMessage::Flush() at ??:0
@ 0x7f191fa7eee9 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x40b4a5 kudu::MasterStressTest::ReplaceTabletThread() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/master-stress-test.cc:393
@ 0x4472f4 std::__invoke_impl<>() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/invoke.h:73
@ 0x4471ad std::__invoke<>() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/invoke.h:95
@ 0x44716a _ZNSt6thread8_InvokerISt5tupleIJMN4kudu16MasterStressTestEFvvEPNS2_26MasterStressTest_Test_TestEEEE9_M_invokeIJLm0ELm1EEEEDTclsr3stdE8__invokespcl10_S_declvalIXT_EEEEESt12_Index_tupleIJXspT_EEE at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:234
@ 0x44711e std::thread::_Invoker<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:243
@ 0x446e59 std::thread::_State_impl<>::_M_run() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:186
@ 0x7f191d3bf6df (unknown) at ??:0
@ 0x7f1923f5e6db start_thread at ??:0
@ 0x7f191ce1861f clone at ??:0
W20260810 02:03:00.908432 23889 consensus_peers.cc:597] T 16db112abfb34d4a9f2ecbee98c39d8b P 48e33bc1e8e94d4887bb64214d0b6c36 -> Peer 50d44445234b4931aaa28141cdaa67ab (127.22.69.1:45491): Couldn't send request to peer 50d44445234b4931aaa28141cdaa67ab. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16db112abfb34d4a9f2ecbee98c39d8b. This is attempt 161: this message will repeat every 5th retry.