Note: This is test shard 3 of 3.
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from CatalogManagerConfigurations/MasterStressTest
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/2
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260811 01:30:37.626520 5551 test_util.cc:268] Using random seed: 1795233891
2026-08-11T01:30:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:30:37Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskZLXb2N/thirdparty/src/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskZLXb2N/thirdparty/src/hadoop-3.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/security.properties
2026-08-11 01:30:44: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskZLXb2N/thirdparty/src/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskZLXb2N/thirdparty/src/hadoop-3.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
I20260811 01:30:52.683512 5551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZLXb2N/build/asan/bin/kudu
/tmp/dist-test-taskZLXb2N/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.107.254:33755
--webserver_interface=127.5.107.254
--webserver_port=0
--builtin_ntp_servers=127.5.107.212:36421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519
--hive_metastore_uris=thrift://127.0.0.1:36237
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260811 01:30:53.075690 5976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:30:53.076030 5976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:30:53.076170 5976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:30:53.084110 5976 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:30:53.084259 5976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:30:53.084376 5976 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:30:53.084466 5976 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:30:53.094733 5976 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.107.212:36421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:36237
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.107.254:33755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.107.254
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:30:53.096288 5976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:30:53.098353 5976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:30:53.111901 5982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:30:53.112002 5986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:30:53.111937 5984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:30:53.113160 5976 server_base.cc:1034] running on GCE node
I20260811 01:30:53.114351 5976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:30:53.116223 5976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:30:53.117583 5976 hybrid_clock.cc:648] HybridClock initialized: now 1786411853117493 us; error 156 us; skew 500 ppm
I20260811 01:30:53.118098 5976 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:30:53.121382 5976 webserver.cc:460] Webserver started at http://127.5.107.254:45627/ using document root <none> and password file <none>
I20260811 01:30:53.122417 5976 fs_manager.cc:359] Metadata directory not provided
I20260811 01:30:53.122623 5976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:30:53.123039 5976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:30:53.128681 5976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/data/instance:
uuid: "61c27cf3d4c846748ae6752e0fa652b9"
format_stamp: "Formatted at 2026-08-11 01:30:53 on dist-test-slave-2p8l"
I20260811 01:30:53.129741 5976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/wal/instance:
uuid: "61c27cf3d4c846748ae6752e0fa652b9"
format_stamp: "Formatted at 2026-08-11 01:30:53 on dist-test-slave-2p8l"
I20260811 01:30:53.136673 5976 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 01:30:53.141017 5998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:30:53.142726 5976 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 01:30:53.143002 5976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/data,/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/wal
uuid: "61c27cf3d4c846748ae6752e0fa652b9"
format_stamp: "Formatted at 2026-08-11 01:30:53 on dist-test-slave-2p8l"
I20260811 01:30:53.143649 5976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:30:53.164999 5976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:30:53.165989 5976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:30:53.166417 5976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:30:53.188683 5976 rpc_server.cc:307] RPC server started. Bound to: 127.5.107.254:33755
I20260811 01:30:53.188725 6090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.107.254:33755 every 8 connection(s)
I20260811 01:30:53.190434 5976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/data/info.pb
I20260811 01:30:53.194659 6092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:30:53.199324 5551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZLXb2N/build/asan/bin/kudu as pid 5976
I20260811 01:30:53.199664 5551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/wal/instance
I20260811 01:30:53.203194 5551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZLXb2N/build/asan/bin/kudu
/tmp/dist-test-taskZLXb2N/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.107.253:36427
--webserver_interface=127.5.107.253
--webserver_port=0
--builtin_ntp_servers=127.5.107.212:36421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519
--hive_metastore_uris=thrift://127.0.0.1:36237
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260811 01:30:53.203011 6092 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } has no permanent_uuid. Determining permanent_uuid...
I20260811 01:30:53.224635 6092 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } has no permanent_uuid. Determining permanent_uuid...
W20260811 01:30:53.226311 6004 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:30:53.228367 6092 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
I20260811 01:30:53.277827 6092 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } attempt: 1
W20260811 01:30:53.280217 6092 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
I20260811 01:30:53.348693 6092 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } attempt: 2
W20260811 01:30:53.351118 6092 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
I20260811 01:30:53.442444 6092 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } attempt: 3
W20260811 01:30:53.444969 6092 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:30:53.570751 6095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:30:53.571102 6095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:30:53.571183 6095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:30:53.578750 6095 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:30:53.578882 6095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:30:53.578929 6095 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:30:53.578979 6095 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:30:53.588280 6092 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } attempt: 4
I20260811 01:30:53.588901 6095 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.107.212:36421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:36237
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.107.253:36427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.107.253
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:30:53.590651 6095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
W20260811 01:30:53.590996 6092 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
I20260811 01:30:53.592715 6095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:30:53.606124 6108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:30:53.606045 6111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:30:53.606215 6109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:30:53.607302 6095 server_base.cc:1034] running on GCE node
I20260811 01:30:53.608212 6095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:30:53.609802 6095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:30:53.611060 6095 hybrid_clock.cc:648] HybridClock initialized: now 1786411853610975 us; error 66 us; skew 500 ppm
I20260811 01:30:53.611424 6095 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:30:53.614157 6095 webserver.cc:460] Webserver started at http://127.5.107.253:42225/ using document root <none> and password file <none>
I20260811 01:30:53.614900 6095 fs_manager.cc:359] Metadata directory not provided
I20260811 01:30:53.615012 6095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:30:53.615310 6095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:30:53.618264 6095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/data/instance:
uuid: "8970a89e5fd744008240ecd668272a4a"
format_stamp: "Formatted at 2026-08-11 01:30:53 on dist-test-slave-2p8l"
I20260811 01:30:53.619071 6095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/wal/instance:
uuid: "8970a89e5fd744008240ecd668272a4a"
format_stamp: "Formatted at 2026-08-11 01:30:53 on dist-test-slave-2p8l"
I20260811 01:30:53.624853 6095 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 01:30:53.628815 6121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:30:53.630395 6095 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 01:30:53.630616 6095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/data,/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/wal
uuid: "8970a89e5fd744008240ecd668272a4a"
format_stamp: "Formatted at 2026-08-11 01:30:53 on dist-test-slave-2p8l"
I20260811 01:30:53.630813 6095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:30:53.654353 6095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:30:53.655247 6095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:30:53.655534 6095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:30:53.675531 6095 rpc_server.cc:307] RPC server started. Bound to: 127.5.107.253:36427
I20260811 01:30:53.675580 6211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.107.253:36427 every 8 connection(s)
I20260811 01:30:53.676988 6095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/data/info.pb
I20260811 01:30:53.677373 5551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZLXb2N/build/asan/bin/kudu as pid 6095
I20260811 01:30:53.677536 5551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/wal/instance
I20260811 01:30:53.679014 5551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZLXb2N/build/asan/bin/kudu
/tmp/dist-test-taskZLXb2N/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.107.252:45519
--webserver_interface=127.5.107.252
--webserver_port=0
--builtin_ntp_servers=127.5.107.212:36421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519
--hive_metastore_uris=thrift://127.0.0.1:36237
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260811 01:30:53.682120 6212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:30:53.689707 6212 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } has no permanent_uuid. Determining permanent_uuid...
I20260811 01:30:53.706015 6212 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } has no permanent_uuid. Determining permanent_uuid...
I20260811 01:30:53.717584 6212 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } has no permanent_uuid. Determining permanent_uuid...
W20260811 01:30:53.718904 6125 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
W20260811 01:30:53.720597 6212 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.107.252:45519: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:30:53.769968 6212 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } attempt: 1
W20260811 01:30:53.772341 6212 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.107.252:45519: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:30:53.840627 6212 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } attempt: 2
W20260811 01:30:53.842998 6212 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.107.252:45519: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:30:53.890309 6092 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } attempt: 5
I20260811 01:30:53.899215 6092 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } has no permanent_uuid. Determining permanent_uuid...
W20260811 01:30:53.901194 6092 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.107.252:45519: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:30:53.934382 6212 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } attempt: 3
W20260811 01:30:53.936995 6212 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.107.252:45519: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:30:53.952440 6092 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } attempt: 1
W20260811 01:30:53.954725 6092 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.107.252:45519: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:30:54.023005 6092 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } attempt: 2
W20260811 01:30:54.025177 6092 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.107.252:45519: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
W20260811 01:30:54.048435 6214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:30:54.048870 6214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:30:54.049031 6214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:30:54.056824 6214 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:30:54.056948 6214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:30:54.057039 6214 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:30:54.057179 6214 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:30:54.067170 6214 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.107.212:36421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:36237
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.107.252:45519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.107.252
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:30:54.068668 6214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:30:54.070631 6214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 01:30:54.080276 6212 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } attempt: 4
W20260811 01:30:54.083009 6212 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.107.252:45519: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
W20260811 01:30:54.084030 6226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:30:54.084002 6225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:30:54.084072 6229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:30:54.085489 6214 server_base.cc:1034] running on GCE node
I20260811 01:30:54.086460 6214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:30:54.088059 6214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:30:54.089340 6214 hybrid_clock.cc:648] HybridClock initialized: now 1786411854089272 us; error 38 us; skew 500 ppm
I20260811 01:30:54.089776 6214 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:30:54.092376 6214 webserver.cc:460] Webserver started at http://127.5.107.252:39897/ using document root <none> and password file <none>
I20260811 01:30:54.093117 6214 fs_manager.cc:359] Metadata directory not provided
I20260811 01:30:54.093277 6214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:30:54.093621 6214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:30:54.096522 6214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/data/instance:
uuid: "c6f417c29b464762944a721d477b2b13"
format_stamp: "Formatted at 2026-08-11 01:30:54 on dist-test-slave-2p8l"
I20260811 01:30:54.097345 6214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/wal/instance:
uuid: "c6f417c29b464762944a721d477b2b13"
format_stamp: "Formatted at 2026-08-11 01:30:54 on dist-test-slave-2p8l"
I20260811 01:30:54.103340 6214 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 01:30:54.107518 6240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:30:54.109129 6214 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 01:30:54.109382 6214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/data,/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/wal
uuid: "c6f417c29b464762944a721d477b2b13"
format_stamp: "Formatted at 2026-08-11 01:30:54 on dist-test-slave-2p8l"
I20260811 01:30:54.109637 6214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:30:54.124013 6214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:30:54.125000 6214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:30:54.125336 6214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:30:54.131422 6092 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } attempt: 3
W20260811 01:30:54.133868 6092 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.107.252:45519: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:30:54.146180 6214 rpc_server.cc:307] RPC server started. Bound to: 127.5.107.252:45519
I20260811 01:30:54.146219 6325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.107.252:45519 every 8 connection(s)
I20260811 01:30:54.147698 6214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/data/info.pb
I20260811 01:30:54.151755 6326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:30:54.152871 5551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZLXb2N/build/asan/bin/kudu as pid 6214
I20260811 01:30:54.153090 5551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/wal/instance
I20260811 01:30:54.159153 6326 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } has no permanent_uuid. Determining permanent_uuid...
I20260811 01:30:54.190652 6326 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } has no permanent_uuid. Determining permanent_uuid...
I20260811 01:30:54.203480 6326 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } has no permanent_uuid. Determining permanent_uuid...
I20260811 01:30:54.215152 6326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13: Bootstrap starting.
I20260811 01:30:54.218999 6326 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13: Neither blocks nor log segments found. Creating new log.
I20260811 01:30:54.220234 6326 log.cc:824] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13: Log is configured to *not* fsync() on all Append() calls
I20260811 01:30:54.223050 6326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13: No bootstrap required, opened a new log
I20260811 01:30:54.232208 6326 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:30:54.232599 6326 raft_consensus.cc:374] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:30:54.232767 6326 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6f417c29b464762944a721d477b2b13, State: Initialized, Role: FOLLOWER
I20260811 01:30:54.233430 6326 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:30:54.234611 6337 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } } }
I20260811 01:30:54.234988 6337 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:30:54.235886 6326 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 01:30:54.254879 6362 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 01:30:54.255064 6362 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 01:30:54.246021 6214 master_runner.cc:186] Error getting master registration for 127.5.107.254:33755: OK, instance_id { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" instance_seqno: 1786411853182321 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 01:30:54.259202 6214 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:30:54.311115 6092 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } attempt: 4
I20260811 01:30:54.331759 6092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9: Bootstrap starting.
I20260811 01:30:54.344430 6092 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9: Neither blocks nor log segments found. Creating new log.
I20260811 01:30:54.345721 6337 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:30:54.346098 6337 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:30:54.346829 6092 log.cc:824] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9: Log is configured to *not* fsync() on all Append() calls
I20260811 01:30:54.348001 6337 leader_election.cc:288] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755), 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427)
I20260811 01:30:54.362275 6092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9: No bootstrap required, opened a new log
I20260811 01:30:54.375296 6092 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:30:54.375782 6092 raft_consensus.cc:374] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:30:54.376003 6092 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61c27cf3d4c846748ae6752e0fa652b9, State: Initialized, Role: FOLLOWER
I20260811 01:30:54.376822 6092 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
W20260811 01:30:54.377688 6249 tablet.cc:2369] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:30:54.383481 6212 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } attempt: 5
I20260811 01:30:54.393277 6092 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:30:54.393311 6049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c6f417c29b464762944a721d477b2b13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61c27cf3d4c846748ae6752e0fa652b9" is_pre_election: true
I20260811 01:30:54.394197 6049 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6f417c29b464762944a721d477b2b13 in term 0.
I20260811 01:30:54.395062 6243 leader_election.cc:302] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9, c6f417c29b464762944a721d477b2b13; no voters:
I20260811 01:30:54.392596 6367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } } }
I20260811 01:30:54.395782 6367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:30:54.395979 6337 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:30:54.396149 6337 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:30:54.396255 6337 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:30:54.399983 6337 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:30:54.401556 6049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c6f417c29b464762944a721d477b2b13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61c27cf3d4c846748ae6752e0fa652b9"
I20260811 01:30:54.401811 6049 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:30:54.405452 6049 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6f417c29b464762944a721d477b2b13 in term 1.
I20260811 01:30:54.406186 6243 leader_election.cc:302] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9, c6f417c29b464762944a721d477b2b13; no voters:
I20260811 01:30:54.407599 6212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a: Bootstrap starting.
I20260811 01:30:54.407790 6337 leader_election.cc:288] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 1 election: Requested vote from peers 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755), 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427)
I20260811 01:30:54.408865 6337 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:30:54.409585 6337 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 1 LEADER]: Becoming Leader. State: Replica: c6f417c29b464762944a721d477b2b13, State: Running, Role: LEADER
I20260811 01:30:54.411957 6337 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
W20260811 01:30:54.419688 6009 tablet.cc:2369] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:30:54.412873 6212 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a: Neither blocks nor log segments found. Creating new log.
I20260811 01:30:54.429224 6212 log.cc:824] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a: Log is configured to *not* fsync() on all Append() calls
I20260811 01:30:54.436785 6212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a: No bootstrap required, opened a new log
I20260811 01:30:54.442754 6368 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6f417c29b464762944a721d477b2b13. Latest consensus state: current_term: 1 leader_uuid: "c6f417c29b464762944a721d477b2b13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } } }
I20260811 01:30:54.445698 6368 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [sys.catalog]: This master's current role is: LEADER
W20260811 01:30:54.450223 6243 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.107.254:33755, user_credentials={real_user=slave}} blocked reactor thread for 44117.5us
I20260811 01:30:54.451236 6212 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:30:54.451874 6212 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:30:54.452095 6212 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8970a89e5fd744008240ecd668272a4a, State: Initialized, Role: FOLLOWER
I20260811 01:30:54.453122 6212 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:30:54.454725 6381 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:30:54.460829 6381 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:30:54.483907 6386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } } }
I20260811 01:30:54.484297 6049 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 1 FOLLOWER]: Refusing update from remote peer c6f417c29b464762944a721d477b2b13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:30:54.484443 6386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:30:54.485745 6212 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:30:54.485618 6368 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:30:54.486928 6160 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:30:54.491958 6164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c6f417c29b464762944a721d477b2b13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8970a89e5fd744008240ecd668272a4a" is_pre_election: true
I20260811 01:30:54.493094 6161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c6f417c29b464762944a721d477b2b13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8970a89e5fd744008240ecd668272a4a"
I20260811 01:30:54.493728 6160 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 1 FOLLOWER]: Refusing update from remote peer c6f417c29b464762944a721d477b2b13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:30:54.495249 6368 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:30:54.529541 6367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6f417c29b464762944a721d477b2b13. Latest consensus state: current_term: 1 leader_uuid: "c6f417c29b464762944a721d477b2b13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } } }
I20260811 01:30:54.530030 6367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:30:54.559414 6367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c6f417c29b464762944a721d477b2b13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } } }
I20260811 01:30:54.559861 6367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:30:54.563977 6337 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c6f417c29b464762944a721d477b2b13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } } }
I20260811 01:30:54.564558 6337 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [sys.catalog]: This master's current role is: LEADER
I20260811 01:30:54.564909 6396 mvcc.cc:205] Tried to move back new op lower bound from 7317142955934220288 to 7317142955697934336. Current Snapshot: MvccSnapshot[applied={T|T < 7317142955934220288 or (T in {7317142955934220288})}]
I20260811 01:30:54.562896 6386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6f417c29b464762944a721d477b2b13. Latest consensus state: current_term: 1 leader_uuid: "c6f417c29b464762944a721d477b2b13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } } }
I20260811 01:30:54.565485 6386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:30:54.583446 6368 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c6f417c29b464762944a721d477b2b13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } } }
I20260811 01:30:54.583771 6368 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [sys.catalog]: This master's current role is: LEADER
I20260811 01:30:54.585587 6381 catalog_manager.cc:1347] Generated new cluster ID: 2f0039b0f9ec4f479f59cf7d8b74b7a2
I20260811 01:30:54.585716 6381 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:30:54.591784 6386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c6f417c29b464762944a721d477b2b13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } } }
I20260811 01:30:54.592104 6386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:30:54.615218 6381 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:30:54.617237 6381 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:30:54.622464 6431 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 01:30:54.627354 5976 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: OK, instance_id { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" instance_seqno: 1786411853669808 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 01:30:54.638983 5976 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:30:54.640946 6381 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13: Generated new TSK 0
I20260811 01:30:54.642285 6381 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 01:30:54.643834 6381 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260811 01:30:54.645056 6381 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260811 01:30:54.655586 5551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZLXb2N/build/asan/bin/kudu
/tmp/dist-test-taskZLXb2N/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.107.193:0
--local_ip_for_outbound_sockets=127.5.107.193
--webserver_interface=127.5.107.193
--webserver_port=0
--tserver_master_addrs=127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519
--builtin_ntp_servers=127.5.107.212:36421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260811 01:30:54.716065 6445 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260811 01:30:55.106012 6439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:30:55.106401 6439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:30:55.106576 6439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:30:55.114670 6439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:30:55.114923 6439 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.107.193
I20260811 01:30:55.124979 6439 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.107.212:36421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.107.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.107.193
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.107.193
--log_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:30:55.126580 6439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:30:55.128486 6439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 01:30:55.141579 6459 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c6f417c29b464762944a721d477b2b13)
I20260811 01:30:55.141891 6459 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
W20260811 01:30:55.143831 6453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:30:55.143959 6458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:30:55.144474 6048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8970a89e5fd744008240ecd668272a4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "61c27cf3d4c846748ae6752e0fa652b9" is_pre_election: true
W20260811 01:30:55.144826 6456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:30:55.145408 6459 leader_election.cc:288] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
I20260811 01:30:55.147012 6283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8970a89e5fd744008240ecd668272a4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c6f417c29b464762944a721d477b2b13" is_pre_election: true
I20260811 01:30:55.147442 6439 server_base.cc:1034] running on GCE node
I20260811 01:30:55.148032 6125 leader_election.cc:302] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8970a89e5fd744008240ecd668272a4a; no voters: 61c27cf3d4c846748ae6752e0fa652b9, c6f417c29b464762944a721d477b2b13
I20260811 01:30:55.148582 6439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:30:55.148569 6459 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:30:55.149799 6439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:30:55.151098 6439 hybrid_clock.cc:648] HybridClock initialized: now 1786411855151015 us; error 43 us; skew 500 ppm
I20260811 01:30:55.151507 6439 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:30:55.159060 6439 webserver.cc:460] Webserver started at http://127.5.107.193:44461/ using document root <none> and password file <none>
I20260811 01:30:55.160184 6439 fs_manager.cc:359] Metadata directory not provided
I20260811 01:30:55.160360 6439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:30:55.160768 6439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:30:55.165176 6439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-0/data/instance:
uuid: "4a331de1488e4a0986b490917efea000"
format_stamp: "Formatted at 2026-08-11 01:30:55 on dist-test-slave-2p8l"
I20260811 01:30:55.166348 6439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-0/wal/instance:
uuid: "4a331de1488e4a0986b490917efea000"
format_stamp: "Formatted at 2026-08-11 01:30:55 on dist-test-slave-2p8l"
I20260811 01:30:55.173750 6439 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260811 01:30:55.178126 6467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:30:55.179960 6439 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 01:30:55.180147 6439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-0/wal
uuid: "4a331de1488e4a0986b490917efea000"
format_stamp: "Formatted at 2026-08-11 01:30:55 on dist-test-slave-2p8l"
I20260811 01:30:55.180351 6439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:30:55.207257 6439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:30:55.208233 6439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:30:55.208598 6439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:30:55.209650 6439 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:30:55.211436 6439 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:30:55.211512 6439 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:30:55.211691 6439 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:30:55.211759 6439 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:30:55.250291 6439 rpc_server.cc:307] RPC server started. Bound to: 127.5.107.193:43339
I20260811 01:30:55.250329 6652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.107.193:43339 every 8 connection(s)
I20260811 01:30:55.251705 6439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-0/data/info.pb
I20260811 01:30:55.259078 5551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZLXb2N/build/asan/bin/kudu as pid 6439
I20260811 01:30:55.259284 5551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-0/wal/instance
I20260811 01:30:55.271032 5551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZLXb2N/build/asan/bin/kudu
/tmp/dist-test-taskZLXb2N/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.107.194:0
--local_ip_for_outbound_sockets=127.5.107.194
--webserver_interface=127.5.107.194
--webserver_port=0
--tserver_master_addrs=127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519
--builtin_ntp_servers=127.5.107.212:36421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260811 01:30:55.306949 6654 heartbeater.cc:347] Connected to a master server at 127.5.107.252:45519
I20260811 01:30:55.307126 6657 heartbeater.cc:347] Connected to a master server at 127.5.107.254:33755
I20260811 01:30:55.307336 6654 heartbeater.cc:464] Registering TS with master...
I20260811 01:30:55.307343 6657 heartbeater.cc:464] Registering TS with master...
I20260811 01:30:55.307687 6658 heartbeater.cc:347] Connected to a master server at 127.5.107.253:36427
I20260811 01:30:55.307823 6658 heartbeater.cc:464] Registering TS with master...
I20260811 01:30:55.308653 6658 heartbeater.cc:511] Master 127.5.107.253:36427 requested a full tablet report, sending...
I20260811 01:30:55.309943 6654 heartbeater.cc:511] Master 127.5.107.252:45519 requested a full tablet report, sending...
I20260811 01:30:55.310413 6657 heartbeater.cc:511] Master 127.5.107.254:33755 requested a full tablet report, sending...
I20260811 01:30:55.312633 6025 ts_manager.cc:194] Registered new tserver with Master: 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
I20260811 01:30:55.315814 6146 ts_manager.cc:194] Registered new tserver with Master: 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
I20260811 01:30:55.315989 6266 ts_manager.cc:194] Registered new tserver with Master: 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
I20260811 01:30:55.317754 6266 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.107.193:54329
I20260811 01:30:55.601176 6672 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c6f417c29b464762944a721d477b2b13)
I20260811 01:30:55.601441 6672 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:30:55.602946 6672 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
I20260811 01:30:55.603699 6160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "61c27cf3d4c846748ae6752e0fa652b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8970a89e5fd744008240ecd668272a4a" is_pre_election: true
I20260811 01:30:55.603754 6283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "61c27cf3d4c846748ae6752e0fa652b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c6f417c29b464762944a721d477b2b13" is_pre_election: true
I20260811 01:30:55.604578 6004 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:30:55.605023 6672 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:30:55.628535 6431 catalog_manager.cc:1259] Loaded cluster ID: 2f0039b0f9ec4f479f59cf7d8b74b7a2
I20260811 01:30:55.628750 6431 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9: loading cluster ID for follower catalog manager: success
I20260811 01:30:55.633123 6431 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9: acquiring CA information for follower catalog manager: success
I20260811 01:30:55.635977 6431 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 01:30:55.659766 6668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:30:55.660071 6668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:30:55.660238 6668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:30:55.668830 6668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:30:55.669128 6668 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.107.194
I20260811 01:30:55.680146 6668 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.107.212:36421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.107.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.107.194
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.107.194
--log_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:30:55.681753 6668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:30:55.683661 6668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:30:55.700043 6683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:30:55.700068 6682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:30:55.701495 6685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:30:55.703684 6668 server_base.cc:1034] running on GCE node
I20260811 01:30:55.704519 6668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:30:55.705718 6668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:30:55.707005 6668 hybrid_clock.cc:648] HybridClock initialized: now 1786411855706917 us; error 59 us; skew 500 ppm
I20260811 01:30:55.707341 6668 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:30:55.710134 6668 webserver.cc:460] Webserver started at http://127.5.107.194:39605/ using document root <none> and password file <none>
I20260811 01:30:55.710899 6668 fs_manager.cc:359] Metadata directory not provided
I20260811 01:30:55.711071 6668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:30:55.711457 6668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:30:55.714543 6668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-1/data/instance:
uuid: "10a02c605db44ec69c67a34f53c2a055"
format_stamp: "Formatted at 2026-08-11 01:30:55 on dist-test-slave-2p8l"
I20260811 01:30:55.715381 6668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-1/wal/instance:
uuid: "10a02c605db44ec69c67a34f53c2a055"
format_stamp: "Formatted at 2026-08-11 01:30:55 on dist-test-slave-2p8l"
I20260811 01:30:55.721942 6445 catalog_manager.cc:1259] Loaded cluster ID: 2f0039b0f9ec4f479f59cf7d8b74b7a2
I20260811 01:30:55.722214 6445 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a: loading cluster ID for follower catalog manager: success
I20260811 01:30:55.727784 6445 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a: acquiring CA information for follower catalog manager: success
I20260811 01:30:55.731154 6445 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 01:30:55.732815 6668 fs_manager.cc:691] Time spent creating directory manager: real 0.017s user 0.008s sys 0.008s
I20260811 01:30:55.737479 6695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:30:55.739396 6668 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 01:30:55.739632 6668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-1/wal
uuid: "10a02c605db44ec69c67a34f53c2a055"
format_stamp: "Formatted at 2026-08-11 01:30:55 on dist-test-slave-2p8l"
I20260811 01:30:55.739926 6668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:30:55.755570 6668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:30:55.756538 6668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:30:55.756910 6668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:30:55.758136 6668 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:30:55.760849 6668 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:30:55.760978 6668 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:30:55.761153 6668 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:30:55.761247 6668 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:30:55.811499 6668 rpc_server.cc:307] RPC server started. Bound to: 127.5.107.194:32971
I20260811 01:30:55.811676 6889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.107.194:32971 every 8 connection(s)
I20260811 01:30:55.813429 6668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-1/data/info.pb
I20260811 01:30:55.821014 6654 heartbeater.cc:503] Master 127.5.107.252:45519 was elected leader, sending a full tablet report...
I20260811 01:30:55.822878 5551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZLXb2N/build/asan/bin/kudu as pid 6668
I20260811 01:30:55.823206 5551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-1/wal/instance
I20260811 01:30:55.829636 5551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZLXb2N/build/asan/bin/kudu
/tmp/dist-test-taskZLXb2N/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.107.195:0
--local_ip_for_outbound_sockets=127.5.107.195
--webserver_interface=127.5.107.195
--webserver_port=0
--tserver_master_addrs=127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519
--builtin_ntp_servers=127.5.107.212:36421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260811 01:30:55.869626 6891 heartbeater.cc:347] Connected to a master server at 127.5.107.254:33755
I20260811 01:30:55.870095 6891 heartbeater.cc:464] Registering TS with master...
I20260811 01:30:55.871004 6890 heartbeater.cc:347] Connected to a master server at 127.5.107.252:45519
I20260811 01:30:55.871169 6890 heartbeater.cc:464] Registering TS with master...
I20260811 01:30:55.871399 6891 heartbeater.cc:511] Master 127.5.107.254:33755 requested a full tablet report, sending...
I20260811 01:30:55.871627 6890 heartbeater.cc:511] Master 127.5.107.252:45519 requested a full tablet report, sending...
I20260811 01:30:55.873174 6266 ts_manager.cc:194] Registered new tserver with Master: 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971)
I20260811 01:30:55.874214 6025 ts_manager.cc:194] Registered new tserver with Master: 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971)
I20260811 01:30:55.874449 6266 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.107.194:33463
I20260811 01:30:55.881325 6896 heartbeater.cc:347] Connected to a master server at 127.5.107.253:36427
I20260811 01:30:55.881513 6896 heartbeater.cc:464] Registering TS with master...
I20260811 01:30:55.882035 6896 heartbeater.cc:511] Master 127.5.107.253:36427 requested a full tablet report, sending...
I20260811 01:30:55.883585 6146 ts_manager.cc:194] Registered new tserver with Master: 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971)
I20260811 01:30:56.191747 6902 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c6f417c29b464762944a721d477b2b13)
I20260811 01:30:56.192106 6902 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:30:56.193608 6048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8970a89e5fd744008240ecd668272a4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "61c27cf3d4c846748ae6752e0fa652b9" is_pre_election: true
I20260811 01:30:56.195348 6283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8970a89e5fd744008240ecd668272a4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c6f417c29b464762944a721d477b2b13" is_pre_election: true
I20260811 01:30:56.196151 6125 leader_election.cc:302] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8970a89e5fd744008240ecd668272a4a; no voters: 61c27cf3d4c846748ae6752e0fa652b9, c6f417c29b464762944a721d477b2b13
I20260811 01:30:56.197479 6902 leader_election.cc:288] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
I20260811 01:30:56.197919 6902 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260811 01:30:56.294775 6901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:30:56.295202 6901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:30:56.295390 6901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:30:56.304369 6901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:30:56.304658 6901 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.107.195
I20260811 01:30:56.317751 6901 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.107.212:36421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.107.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.107.195
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.107.195
--log_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:30:56.319696 6901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:30:56.322952 6901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:30:56.341409 6909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:30:56.341409 6910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:30:56.341643 6912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:30:56.342485 6901 server_base.cc:1034] running on GCE node
I20260811 01:30:56.343904 6901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:30:56.345252 6901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:30:56.346549 6901 hybrid_clock.cc:648] HybridClock initialized: now 1786411856346446 us; error 55 us; skew 500 ppm
I20260811 01:30:56.347033 6901 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:30:56.350585 6901 webserver.cc:460] Webserver started at http://127.5.107.195:34281/ using document root <none> and password file <none>
I20260811 01:30:56.351503 6901 fs_manager.cc:359] Metadata directory not provided
I20260811 01:30:56.351693 6901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:30:56.352097 6901 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:30:56.355443 6901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-2/data/instance:
uuid: "992432fc73c94fcc9f2142b27ef725af"
format_stamp: "Formatted at 2026-08-11 01:30:56 on dist-test-slave-2p8l"
I20260811 01:30:56.356424 6901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-2/wal/instance:
uuid: "992432fc73c94fcc9f2142b27ef725af"
format_stamp: "Formatted at 2026-08-11 01:30:56 on dist-test-slave-2p8l"
I20260811 01:30:56.364235 6901 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.002s
I20260811 01:30:56.369215 6918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:30:56.371232 6901 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 01:30:56.371466 6901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-2/wal
uuid: "992432fc73c94fcc9f2142b27ef725af"
format_stamp: "Formatted at 2026-08-11 01:30:56 on dist-test-slave-2p8l"
I20260811 01:30:56.371691 6901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:30:56.377944 6890 heartbeater.cc:503] Master 127.5.107.252:45519 was elected leader, sending a full tablet report...
I20260811 01:30:56.393316 6901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:30:56.394465 6901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:30:56.394801 6901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:30:56.395967 6901 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:30:56.397607 6901 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:30:56.397686 6901 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:30:56.397838 6901 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:30:56.397894 6901 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:30:56.441052 6901 rpc_server.cc:307] RPC server started. Bound to: 127.5.107.195:39433
I20260811 01:30:56.441102 7108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.107.195:39433 every 8 connection(s)
I20260811 01:30:56.442728 6901 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-2/data/info.pb
I20260811 01:30:56.449801 5551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZLXb2N/build/asan/bin/kudu as pid 6901
I20260811 01:30:56.450007 5551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-2/wal/instance
I20260811 01:30:56.475355 7113 heartbeater.cc:347] Connected to a master server at 127.5.107.253:36427
I20260811 01:30:56.475777 7113 heartbeater.cc:464] Registering TS with master...
I20260811 01:30:56.477916 7113 heartbeater.cc:511] Master 127.5.107.253:36427 requested a full tablet report, sending...
I20260811 01:30:56.479218 7111 heartbeater.cc:347] Connected to a master server at 127.5.107.254:33755
I20260811 01:30:56.479394 7111 heartbeater.cc:464] Registering TS with master...
I20260811 01:30:56.479965 7111 heartbeater.cc:511] Master 127.5.107.254:33755 requested a full tablet report, sending...
I20260811 01:30:56.480382 6146 ts_manager.cc:194] Registered new tserver with Master: 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
I20260811 01:30:56.481339 6025 ts_manager.cc:194] Registered new tserver with Master: 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
I20260811 01:30:56.483978 7110 heartbeater.cc:347] Connected to a master server at 127.5.107.252:45519
I20260811 01:30:56.484164 7110 heartbeater.cc:464] Registering TS with master...
I20260811 01:30:56.484795 7110 heartbeater.cc:511] Master 127.5.107.252:45519 requested a full tablet report, sending...
I20260811 01:30:56.486243 6266 ts_manager.cc:194] Registered new tserver with Master: 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
I20260811 01:30:56.487360 6266 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.107.195:46481
I20260811 01:30:56.501555 5551 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 01:30:56.536717 6266 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46938:
name: "default.Table_d472cfe99d004f68bc6beda9b7c20bc4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:30:56.540536 6263 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46938:
name: "default.Table_977e759aa4644f0091ecc8c4613cffc2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:30:56.543641 6262 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46938:
name: "default.Table_071a47d589c9407aa47c9d668c44cc23"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:30:56.544472 6260 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46938:
name: "default.Table_f7f5b09418c242d092b1948b866994e5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:30:56.546115 6259 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46938:
name: "default.Table_16925b327c134e08987029291b57034c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:30:56.547307 6255 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46938:
name: "default.Table_b6731279f2fb4ef2a0a845c55840c58b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:30:56.551910 6251 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46938:
name: "default.Table_4fad8fd88a6444b2a3dbc99d87835413"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:30:56.553392 6252 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46938:
name: "default.Table_c93618b4070e4974a6a794087d73fa73"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 01:30:56.554049 6266 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_d472cfe99d004f68bc6beda9b7c20bc4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 01:30:56.554011 6254 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46938:
name: "default.Table_34b4086ff0f64632adbac74648d9874a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:30:56.553122 6253 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46938:
name: "default.Table_1354313d091b46d88f34c2ccd44bf8e0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:30:56.578181 5551 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZLXb2N/build/asan/bin/kudu pid 6214
I20260811 01:30:56.579741 6316 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:30:56.737943 5551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZLXb2N/build/asan/bin/kudu with pid 6214
W20260811 01:30:56.779971 6124 connection.cc:535] server connection from 127.0.0.1:50320 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 01:30:56.780206 7131 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.5.107.252:45519)
W20260811 01:30:56.827821 6654 heartbeater.cc:644] Failed to heartbeat to 127.5.107.252:45519 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
W20260811 01:30:56.883119 6890 heartbeater.cc:644] Failed to heartbeat to 127.5.107.252:45519 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:30:56.970486 7197 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c6f417c29b464762944a721d477b2b13)
I20260811 01:30:56.970796 7197 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:30:56.971583 7197 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
I20260811 01:30:56.972249 6160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "61c27cf3d4c846748ae6752e0fa652b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8970a89e5fd744008240ecd668272a4a" is_pre_election: true
W20260811 01:30:56.975508 6004 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:30:56.975771 6004 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:30:56.976118 7197 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:30:56.989907 7110 heartbeater.cc:503] Master 127.5.107.252:45519 was elected leader, sending a full tablet report...
W20260811 01:30:56.991628 7110 heartbeater.cc:644] Failed to heartbeat to 127.5.107.252:45519 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:30:57.239009 7201 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c6f417c29b464762944a721d477b2b13)
I20260811 01:30:57.239287 7201 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:30:57.240116 7201 leader_election.cc:288] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
I20260811 01:30:57.240748 6048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8970a89e5fd744008240ecd668272a4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "61c27cf3d4c846748ae6752e0fa652b9" is_pre_election: true
I20260811 01:30:57.241025 6048 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8970a89e5fd744008240ecd668272a4a in term 1.
I20260811 01:30:57.241626 6124 leader_election.cc:302] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9, 8970a89e5fd744008240ecd668272a4a; no voters:
I20260811 01:30:57.241977 7201 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 01:30:57.242173 7201 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 1 FOLLOWER]: Starting leader election (detected failure of leader c6f417c29b464762944a721d477b2b13)
I20260811 01:30:57.242311 7201 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 1 FOLLOWER]: Advancing to term 2
W20260811 01:30:57.243043 6125 leader_election.cc:334] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:30:57.299851 7201 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:30:57.302009 7201 leader_election.cc:288] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [CANDIDATE]: Term 2 election: Requested vote from peers 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
I20260811 01:30:57.303202 6048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8970a89e5fd744008240ecd668272a4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "61c27cf3d4c846748ae6752e0fa652b9"
I20260811 01:30:57.303488 6048 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 1 FOLLOWER]: Advancing to term 2
W20260811 01:30:57.303627 6125 leader_election.cc:334] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:30:57.399363 6048 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8970a89e5fd744008240ecd668272a4a in term 2.
I20260811 01:30:57.400218 6124 leader_election.cc:302] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9, 8970a89e5fd744008240ecd668272a4a; no voters: c6f417c29b464762944a721d477b2b13
I20260811 01:30:57.400899 7201 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 2 FOLLOWER]: Leader election won for term 2
I20260811 01:30:57.407685 7201 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 2 LEADER]: Becoming Leader. State: Replica: 8970a89e5fd744008240ecd668272a4a, State: Running, Role: LEADER
I20260811 01:30:57.408475 7201 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:30:57.412132 7207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8970a89e5fd744008240ecd668272a4a. Latest consensus state: current_term: 2 leader_uuid: "8970a89e5fd744008240ecd668272a4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } } }
I20260811 01:30:57.412753 7207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [sys.catalog]: This master's current role is: LEADER
I20260811 01:30:57.413707 7210 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:30:57.415809 7210 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:30:57.416674 7210 catalog_manager.cc:1259] Loaded cluster ID: 2f0039b0f9ec4f479f59cf7d8b74b7a2
I20260811 01:30:57.416796 7210 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:30:57.418277 7210 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:30:57.419139 7210 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a: Loaded TSK: 0
I20260811 01:30:57.420192 7210 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 01:30:57.421016 7210 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260811 01:30:57.421803 7210 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260811 01:30:57.449883 6131 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34732:
name: "default.Table_16925b327c134e08987029291b57034c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:30:57.450138 6135 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34732:
name: "default.Table_d472cfe99d004f68bc6beda9b7c20bc4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:30:57.449615 6132 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34732:
name: "default.Table_f7f5b09418c242d092b1948b866994e5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:30:57.449949 6143 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34732:
name: "default.Table_c93618b4070e4974a6a794087d73fa73"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:30:57.449997 6130 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34732:
name: "default.Table_b6731279f2fb4ef2a0a845c55840c58b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:30:57.449575 6140 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34732:
name: "default.Table_977e759aa4644f0091ecc8c4613cffc2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:30:57.450165 6133 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34732:
name: "default.Table_071a47d589c9407aa47c9d668c44cc23"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:30:57.449237 6141 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34732:
name: "default.Table_34b4086ff0f64632adbac74648d9874a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:30:57.449892 6142 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34732:
name: "default.Table_1354313d091b46d88f34c2ccd44bf8e0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:30:57.447646 6146 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34732:
name: "default.Table_4fad8fd88a6444b2a3dbc99d87835413"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 01:30:57.468161 6132 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_f7f5b09418c242d092b1948b866994e5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 01:30:57.468235 6143 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_c93618b4070e4974a6a794087d73fa73 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 01:30:57.469051 6135 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_d472cfe99d004f68bc6beda9b7c20bc4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 01:30:57.674916 5551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZLXb2N/build/asan/bin/kudu
/tmp/dist-test-taskZLXb2N/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.107.252:45519
--webserver_interface=127.5.107.252
--webserver_port=39897
--builtin_ntp_servers=127.5.107.212:36421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519
--hive_metastore_uris=thrift://127.0.0.1:36237
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260811 01:30:57.816960 6143 catalog_manager.cc:2148] created HMS catalog entry for table [id=a65d01fd84fa41a3b31bf35b60f5cbaf]
I20260811 01:30:57.827837 6048 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 2 FOLLOWER]: Refusing update from remote peer 8970a89e5fd744008240ecd668272a4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20260811 01:30:57.829417 7201 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260811 01:30:57.875303 7222 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8970a89e5fd744008240ecd668272a4a. Latest consensus state: current_term: 2 leader_uuid: "8970a89e5fd744008240ecd668272a4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } } }
I20260811 01:30:57.876072 7222 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:30:57.878767 7217 mvcc.cc:205] Tried to move back new op lower bound from 7317142969652056064 to 7317142967951904768. Current Snapshot: MvccSnapshot[applied={T|T < 7317142969652056064}]
W20260811 01:30:57.886647 6125 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a -> Peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Couldn't send request to peer c6f417c29b464762944a721d477b2b13. Status: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 01:30:57.904021 7207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "8970a89e5fd744008240ecd668272a4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } } }
I20260811 01:30:57.904482 7207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [sys.catalog]: This master's current role is: LEADER
I20260811 01:30:57.906090 7222 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "8970a89e5fd744008240ecd668272a4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } } }
I20260811 01:30:57.906423 7222 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:30:57.938879 6143 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34732:
table { table_name: "default.Table_c93618b4070e4974a6a794087d73fa73" } new_table_name: "default.Table_9be7f6a4237648ab8b421f62026d3ad5" modify_external_catalogs: true
I20260811 01:30:57.962689 6557 tablet_service.cc:1467] Processing CreateTablet for tablet 4ac81f2f008042de9813bcf42126d77a (DEFAULT_TABLE table=default.table_c93618b4070e4974a6a794087d73fa73 [id=a65d01fd84fa41a3b31bf35b60f5cbaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:30:57.962813 6774 tablet_service.cc:1467] Processing CreateTablet for tablet 4ac81f2f008042de9813bcf42126d77a (DEFAULT_TABLE table=default.table_c93618b4070e4974a6a794087d73fa73 [id=a65d01fd84fa41a3b31bf35b60f5cbaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:30:57.964373 6774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ac81f2f008042de9813bcf42126d77a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:30:57.964392 6557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ac81f2f008042de9813bcf42126d77a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:30:57.971931 7009 tablet_service.cc:1467] Processing CreateTablet for tablet 4ac81f2f008042de9813bcf42126d77a (DEFAULT_TABLE table=default.table_c93618b4070e4974a6a794087d73fa73 [id=a65d01fd84fa41a3b31bf35b60f5cbaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:30:57.973516 7009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ac81f2f008042de9813bcf42126d77a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:30:57.989326 7249 tablet_bootstrap.cc:492] T 4ac81f2f008042de9813bcf42126d77a P 10a02c605db44ec69c67a34f53c2a055: Bootstrap starting.
I20260811 01:30:57.989835 7248 tablet_bootstrap.cc:492] T 4ac81f2f008042de9813bcf42126d77a P 4a331de1488e4a0986b490917efea000: Bootstrap starting.
I20260811 01:30:57.994120 7249 tablet_bootstrap.cc:654] T 4ac81f2f008042de9813bcf42126d77a P 10a02c605db44ec69c67a34f53c2a055: Neither blocks nor log segments found. Creating new log.
I20260811 01:30:57.994472 7248 tablet_bootstrap.cc:654] T 4ac81f2f008042de9813bcf42126d77a P 4a331de1488e4a0986b490917efea000: Neither blocks nor log segments found. Creating new log.
I20260811 01:30:57.995857 7249 log.cc:824] T 4ac81f2f008042de9813bcf42126d77a P 10a02c605db44ec69c67a34f53c2a055: Log is configured to *not* fsync() on all Append() calls
I20260811 01:30:57.996052 7248 log.cc:824] T 4ac81f2f008042de9813bcf42126d77a P 4a331de1488e4a0986b490917efea000: Log is configured to *not* fsync() on all Append() calls
I20260811 01:30:58.000384 7252 tablet_bootstrap.cc:492] T 4ac81f2f008042de9813bcf42126d77a P 992432fc73c94fcc9f2142b27ef725af: Bootstrap starting.
I20260811 01:30:58.003592 7248 tablet_bootstrap.cc:492] T 4ac81f2f008042de9813bcf42126d77a P 4a331de1488e4a0986b490917efea000: No bootstrap required, opened a new log
I20260811 01:30:58.004112 7248 ts_tablet_manager.cc:1397] T 4ac81f2f008042de9813bcf42126d77a P 4a331de1488e4a0986b490917efea000: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.003s
I20260811 01:30:58.005175 7249 tablet_bootstrap.cc:492] T 4ac81f2f008042de9813bcf42126d77a P 10a02c605db44ec69c67a34f53c2a055: No bootstrap required, opened a new log
I20260811 01:30:58.005739 7249 ts_tablet_manager.cc:1397] T 4ac81f2f008042de9813bcf42126d77a P 10a02c605db44ec69c67a34f53c2a055: Time spent bootstrapping tablet: real 0.017s user 0.011s sys 0.004s
I20260811 01:30:58.014251 7252 tablet_bootstrap.cc:654] T 4ac81f2f008042de9813bcf42126d77a P 992432fc73c94fcc9f2142b27ef725af: Neither blocks nor log segments found. Creating new log.
I20260811 01:30:58.015733 7252 log.cc:824] T 4ac81f2f008042de9813bcf42126d77a P 992432fc73c94fcc9f2142b27ef725af: Log is configured to *not* fsync() on all Append() calls
I20260811 01:30:58.018208 7249 raft_consensus.cc:348] T 4ac81f2f008042de9813bcf42126d77a P 10a02c605db44ec69c67a34f53c2a055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:30:58.018849 7249 raft_consensus.cc:374] T 4ac81f2f008042de9813bcf42126d77a P 10a02c605db44ec69c67a34f53c2a055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:30:58.019011 7249 raft_consensus.cc:729] T 4ac81f2f008042de9813bcf42126d77a P 10a02c605db44ec69c67a34f53c2a055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10a02c605db44ec69c67a34f53c2a055, State: Initialized, Role: FOLLOWER
I20260811 01:30:58.019886 7249 consensus_queue.cc:260] T 4ac81f2f008042de9813bcf42126d77a P 10a02c605db44ec69c67a34f53c2a055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:30:58.020534 7248 raft_consensus.cc:348] T 4ac81f2f008042de9813bcf42126d77a P 4a331de1488e4a0986b490917efea000 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:30:58.021204 7248 raft_consensus.cc:374] T 4ac81f2f008042de9813bcf42126d77a P 4a331de1488e4a0986b490917efea000 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:30:58.021387 7248 raft_consensus.cc:729] T 4ac81f2f008042de9813bcf42126d77a P 4a331de1488e4a0986b490917efea000 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a331de1488e4a0986b490917efea000, State: Initialized, Role: FOLLOWER
I20260811 01:30:58.022410 7248 consensus_queue.cc:260] T 4ac81f2f008042de9813bcf42126d77a P 4a331de1488e4a0986b490917efea000 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:30:58.025828 7252 tablet_bootstrap.cc:492] T 4ac81f2f008042de9813bcf42126d77a P 992432fc73c94fcc9f2142b27ef725af: No bootstrap required, opened a new log
I20260811 01:30:58.026335 7252 ts_tablet_manager.cc:1397] T 4ac81f2f008042de9813bcf42126d77a P 992432fc73c94fcc9f2142b27ef725af: Time spent bootstrapping tablet: real 0.026s user 0.016s sys 0.000s
I20260811 01:30:58.027304 6132 catalog_manager.cc:2148] created HMS catalog entry for table [id=69f9a5a340894b4bb41b150af86e0d8a]
I20260811 01:30:58.028362 7249 ts_tablet_manager.cc:1428] T 4ac81f2f008042de9813bcf42126d77a P 10a02c605db44ec69c67a34f53c2a055: Time spent starting tablet: real 0.022s user 0.013s sys 0.007s
I20260811 01:30:58.032089 6896 heartbeater.cc:503] Master 127.5.107.253:36427 was elected leader, sending a full tablet report...
I20260811 01:30:58.045198 7009 tablet_service.cc:1467] Processing CreateTablet for tablet 8d33739d8c014d8686784505ff5f13ab (DEFAULT_TABLE table=default.table_f7f5b09418c242d092b1948b866994e5 [id=69f9a5a340894b4bb41b150af86e0d8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:30:58.045559 6132 master_service.cc:946] ReplaceTablet: received request to replace tablet 4ac81f2f008042de9813bcf42126d77a from {username='slave'} at 127.0.0.1:34732
I20260811 01:30:58.045742 7009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d33739d8c014d8686784505ff5f13ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:30:58.047694 6774 tablet_service.cc:1467] Processing CreateTablet for tablet 8d33739d8c014d8686784505ff5f13ab (DEFAULT_TABLE table=default.table_f7f5b09418c242d092b1948b866994e5 [id=69f9a5a340894b4bb41b150af86e0d8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:30:58.048183 6774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d33739d8c014d8686784505ff5f13ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:30:58.055013 6557 tablet_service.cc:1467] Processing CreateTablet for tablet 8d33739d8c014d8686784505ff5f13ab (DEFAULT_TABLE table=default.table_f7f5b09418c242d092b1948b866994e5 [id=69f9a5a340894b4bb41b150af86e0d8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:30:58.055513 6557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d33739d8c014d8686784505ff5f13ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:30:58.056490 6658 heartbeater.cc:503] Master 127.5.107.253:36427 was elected leader, sending a full tablet report...
I20260811 01:30:58.059181 7248 ts_tablet_manager.cc:1428] T 4ac81f2f008042de9813bcf42126d77a P 4a331de1488e4a0986b490917efea000: Time spent starting tablet: real 0.055s user 0.014s sys 0.012s
I20260811 01:30:58.061480 7252 raft_consensus.cc:348] T 4ac81f2f008042de9813bcf42126d77a P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:30:58.062089 7252 raft_consensus.cc:374] T 4ac81f2f008042de9813bcf42126d77a P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:30:58.062234 7252 raft_consensus.cc:729] T 4ac81f2f008042de9813bcf42126d77a P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 992432fc73c94fcc9f2142b27ef725af, State: Initialized, Role: FOLLOWER
I20260811 01:30:58.062969 7252 consensus_queue.cc:260] T 4ac81f2f008042de9813bcf42126d77a P 992432fc73c94fcc9f2142b27ef725af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:30:58.070905 6132 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a: Sending DeleteTablet for 3 replicas of tablet 4ac81f2f008042de9813bcf42126d77a
I20260811 01:30:58.071888 6132 catalog_manager.cc:6565] ReplaceTablet: tablet 4ac81f2f008042de9813bcf42126d77a deleted and replaced by tablet b31aa7bcb4b249bd931573f48ca253de
I20260811 01:30:58.072737 7009 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ac81f2f008042de9813bcf42126d77a with delete_type TABLET_DATA_DELETED (replaced by tablet b31aa7bcb4b249bd931573f48ca253de) from {username='slave'} at 127.0.0.1:39622
I20260811 01:30:58.075749 6556 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ac81f2f008042de9813bcf42126d77a with delete_type TABLET_DATA_DELETED (replaced by tablet b31aa7bcb4b249bd931573f48ca253de) from {username='slave'} at 127.0.0.1:49320
I20260811 01:30:58.076311 6773 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ac81f2f008042de9813bcf42126d77a with delete_type TABLET_DATA_DELETED (replaced by tablet b31aa7bcb4b249bd931573f48ca253de) from {username='slave'} at 127.0.0.1:41446
I20260811 01:30:58.091271 6554 tablet_service.cc:1467] Processing CreateTablet for tablet b31aa7bcb4b249bd931573f48ca253de (DEFAULT_TABLE table=default.table_c93618b4070e4974a6a794087d73fa73 [id=a65d01fd84fa41a3b31bf35b60f5cbaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:30:58.092022 6554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b31aa7bcb4b249bd931573f48ca253de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:30:58.095764 7008 tablet_service.cc:1467] Processing CreateTablet for tablet b31aa7bcb4b249bd931573f48ca253de (DEFAULT_TABLE table=default.table_c93618b4070e4974a6a794087d73fa73 [id=a65d01fd84fa41a3b31bf35b60f5cbaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:30:58.096571 7008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b31aa7bcb4b249bd931573f48ca253de. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 01:30:58.099745 6898 tablet.cc:2369] T 4ac81f2f008042de9813bcf42126d77a P 10a02c605db44ec69c67a34f53c2a055: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:30:58.106895 6132 master_service.cc:946] ReplaceTablet: received request to replace tablet 4ac81f2f008042de9813bcf42126d77a from {username='slave'} at 127.0.0.1:34732
I20260811 01:30:58.107712 6132 master_service.cc:946] ReplaceTablet: received request to replace tablet 4ac81f2f008042de9813bcf42126d77a from {username='slave'} at 127.0.0.1:34732
I20260811 01:30:58.108355 6132 master_service.cc:946] ReplaceTablet: received request to replace tablet 4ac81f2f008042de9813bcf42126d77a from {username='slave'} at 127.0.0.1:34732
I20260811 01:30:58.109303 7269 tablet_replica.cc:330] T 4ac81f2f008042de9813bcf42126d77a P 4a331de1488e4a0986b490917efea000: stopping tablet replica
I20260811 01:30:58.112622 7269 raft_consensus.cc:2229] T 4ac81f2f008042de9813bcf42126d77a P 4a331de1488e4a0986b490917efea000 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 01:30:58.112890 6770 tablet_service.cc:1467] Processing CreateTablet for tablet b31aa7bcb4b249bd931573f48ca253de (DEFAULT_TABLE table=default.table_c93618b4070e4974a6a794087d73fa73 [id=a65d01fd84fa41a3b31bf35b60f5cbaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:30:58.113546 6770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b31aa7bcb4b249bd931573f48ca253de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:30:58.114038 7269 raft_consensus.cc:2258] T 4ac81f2f008042de9813bcf42126d77a P 4a331de1488e4a0986b490917efea000 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260811 01:30:58.115777 6124 catalog_manager.cc:4821] TS 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): delete failed for tablet 4ac81f2f008042de9813bcf42126d77a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4ac81f2f008042de9813bcf42126d77a already in progress: creating tablet
I20260811 01:30:58.109983 6132 master_service.cc:946] ReplaceTablet: received request to replace tablet 4ac81f2f008042de9813bcf42126d77a from {username='slave'} at 127.0.0.1:34732
I20260811 01:30:58.117548 7249 tablet_bootstrap.cc:492] T 8d33739d8c014d8686784505ff5f13ab P 10a02c605db44ec69c67a34f53c2a055: Bootstrap starting.
I20260811 01:30:58.119596 7249 tablet_bootstrap.cc:654] T 8d33739d8c014d8686784505ff5f13ab P 10a02c605db44ec69c67a34f53c2a055: Neither blocks nor log segments found. Creating new log.
I20260811 01:30:58.119709 7269 ts_tablet_manager.cc:1905] T 4ac81f2f008042de9813bcf42126d77a P 4a331de1488e4a0986b490917efea000: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:30:58.120294 6132 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34732:
table { table_name: "default.Table_f7f5b09418c242d092b1948b866994e5" } new_table_name: "default.Table_414082fd726f49d4a5926d9c9c739495" modify_external_catalogs: true
I20260811 01:30:58.120895 7248 tablet_bootstrap.cc:492] T 8d33739d8c014d8686784505ff5f13ab P 4a331de1488e4a0986b490917efea000: Bootstrap starting.
I20260811 01:30:58.122809 7271 tablet_replica.cc:330] T 4ac81f2f008042de9813bcf42126d77a P 10a02c605db44ec69c67a34f53c2a055: stopping tablet replica
I20260811 01:30:58.123255 7248 tablet_bootstrap.cc:654] T 8d33739d8c014d8686784505ff5f13ab P 4a331de1488e4a0986b490917efea000: Neither blocks nor log segments found. Creating new log.
I20260811 01:30:58.123420 7271 raft_consensus.cc:2229] T 4ac81f2f008042de9813bcf42126d77a P 10a02c605db44ec69c67a34f53c2a055 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 01:30:58.123997 7271 raft_consensus.cc:2258] T 4ac81f2f008042de9813bcf42126d77a P 10a02c605db44ec69c67a34f53c2a055 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 01:30:58.131108 7271 ts_tablet_manager.cc:1905] T 4ac81f2f008042de9813bcf42126d77a P 10a02c605db44ec69c67a34f53c2a055: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:30:58.133061 7269 ts_tablet_manager.cc:1918] T 4ac81f2f008042de9813bcf42126d77a P 4a331de1488e4a0986b490917efea000: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 01:30:58.135495 7252 ts_tablet_manager.cc:1428] T 4ac81f2f008042de9813bcf42126d77a P 992432fc73c94fcc9f2142b27ef725af: Time spent starting tablet: real 0.109s user 0.031s sys 0.029s
I20260811 01:30:58.136631 7249 tablet_bootstrap.cc:492] T 8d33739d8c014d8686784505ff5f13ab P 10a02c605db44ec69c67a34f53c2a055: No bootstrap required, opened a new log
I20260811 01:30:58.136869 7249 ts_tablet_manager.cc:1397] T 8d33739d8c014d8686784505ff5f13ab P 10a02c605db44ec69c67a34f53c2a055: Time spent bootstrapping tablet: real 0.019s user 0.010s sys 0.001s
I20260811 01:30:58.137392 7113 heartbeater.cc:503] Master 127.5.107.253:36427 was elected leader, sending a full tablet report...
I20260811 01:30:58.138207 7249 raft_consensus.cc:348] T 8d33739d8c014d8686784505ff5f13ab P 10a02c605db44ec69c67a34f53c2a055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:30:58.138504 7252 tablet_bootstrap.cc:492] T 8d33739d8c014d8686784505ff5f13ab P 992432fc73c94fcc9f2142b27ef725af: Bootstrap starting.
I20260811 01:30:58.138474 7249 raft_consensus.cc:374] T 8d33739d8c014d8686784505ff5f13ab P 10a02c605db44ec69c67a34f53c2a055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:30:58.138608 7249 raft_consensus.cc:729] T 8d33739d8c014d8686784505ff5f13ab P 10a02c605db44ec69c67a34f53c2a055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10a02c605db44ec69c67a34f53c2a055, State: Initialized, Role: FOLLOWER
I20260811 01:30:58.138957 7249 consensus_queue.cc:260] T 8d33739d8c014d8686784505ff5f13ab P 10a02c605db44ec69c67a34f53c2a055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:30:58.140436 7252 tablet_bootstrap.cc:654] T 8d33739d8c014d8686784505ff5f13ab P 992432fc73c94fcc9f2142b27ef725af: Neither blocks nor log segments found. Creating new log.
I20260811 01:30:58.142108 7249 ts_tablet_manager.cc:1428] T 8d33739d8c014d8686784505ff5f13ab P 10a02c605db44ec69c67a34f53c2a055: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260811 01:30:58.145340 7249 tablet_bootstrap.cc:492] T b31aa7bcb4b249bd931573f48ca253de P 10a02c605db44ec69c67a34f53c2a055: Bootstrap starting.
I20260811 01:30:58.143098 7248 tablet_bootstrap.cc:492] T 8d33739d8c014d8686784505ff5f13ab P 4a331de1488e4a0986b490917efea000: No bootstrap required, opened a new log
I20260811 01:30:58.146944 7248 ts_tablet_manager.cc:1397] T 8d33739d8c014d8686784505ff5f13ab P 4a331de1488e4a0986b490917efea000: Time spent bootstrapping tablet: real 0.026s user 0.004s sys 0.006s
I20260811 01:30:58.146701 7269 log.cc:1196] T 4ac81f2f008042de9813bcf42126d77a P 4a331de1488e4a0986b490917efea000: Deleting WAL directory at /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-0/wal/wals/4ac81f2f008042de9813bcf42126d77a
I20260811 01:30:58.148250 7248 raft_consensus.cc:348] T 8d33739d8c014d8686784505ff5f13ab P 4a331de1488e4a0986b490917efea000 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:30:58.148572 7248 raft_consensus.cc:374] T 8d33739d8c014d8686784505ff5f13ab P 4a331de1488e4a0986b490917efea000 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:30:58.148721 7248 raft_consensus.cc:729] T 8d33739d8c014d8686784505ff5f13ab P 4a331de1488e4a0986b490917efea000 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a331de1488e4a0986b490917efea000, State: Initialized, Role: FOLLOWER
I20260811 01:30:58.148799 7269 ts_tablet_manager.cc:1939] T 4ac81f2f008042de9813bcf42126d77a P 4a331de1488e4a0986b490917efea000: Deleting consensus metadata
I20260811 01:30:58.149078 7248 consensus_queue.cc:260] T 8d33739d8c014d8686784505ff5f13ab P 4a331de1488e4a0986b490917efea000 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:30:58.149741 7248 ts_tablet_manager.cc:1428] T 8d33739d8c014d8686784505ff5f13ab P 4a331de1488e4a0986b490917efea000: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260811 01:30:58.150158 7248 tablet_bootstrap.cc:492] T b31aa7bcb4b249bd931573f48ca253de P 4a331de1488e4a0986b490917efea000: Bootstrap starting.
I20260811 01:30:58.152045 7248 tablet_bootstrap.cc:654] T b31aa7bcb4b249bd931573f48ca253de P 4a331de1488e4a0986b490917efea000: Neither blocks nor log segments found. Creating new log.
I20260811 01:30:58.152048 6125 catalog_manager.cc:4841] TS 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): tablet 4ac81f2f008042de9813bcf42126d77a (table default.table_c93618b4070e4974a6a794087d73fa73 [id=a65d01fd84fa41a3b31bf35b60f5cbaf]) successfully deleted
I20260811 01:30:58.152853 7249 tablet_bootstrap.cc:654] T b31aa7bcb4b249bd931573f48ca253de P 10a02c605db44ec69c67a34f53c2a055: Neither blocks nor log segments found. Creating new log.
I20260811 01:30:58.159592 7271 ts_tablet_manager.cc:1918] T 4ac81f2f008042de9813bcf42126d77a P 10a02c605db44ec69c67a34f53c2a055: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 01:30:58.164554 7248 tablet_bootstrap.cc:492] T b31aa7bcb4b249bd931573f48ca253de P 4a331de1488e4a0986b490917efea000: No bootstrap required, opened a new log
I20260811 01:30:58.165515 7248 ts_tablet_manager.cc:1397] T b31aa7bcb4b249bd931573f48ca253de P 4a331de1488e4a0986b490917efea000: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.002s
I20260811 01:30:58.165657 7271 log.cc:1196] T 4ac81f2f008042de9813bcf42126d77a P 10a02c605db44ec69c67a34f53c2a055: Deleting WAL directory at /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-1/wal/wals/4ac81f2f008042de9813bcf42126d77a
I20260811 01:30:58.166301 7271 ts_tablet_manager.cc:1939] T 4ac81f2f008042de9813bcf42126d77a P 10a02c605db44ec69c67a34f53c2a055: Deleting consensus metadata
I20260811 01:30:58.166536 7248 raft_consensus.cc:348] T b31aa7bcb4b249bd931573f48ca253de P 4a331de1488e4a0986b490917efea000 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:30:58.166963 7248 raft_consensus.cc:374] T b31aa7bcb4b249bd931573f48ca253de P 4a331de1488e4a0986b490917efea000 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:30:58.167114 7248 raft_consensus.cc:729] T b31aa7bcb4b249bd931573f48ca253de P 4a331de1488e4a0986b490917efea000 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a331de1488e4a0986b490917efea000, State: Initialized, Role: FOLLOWER
I20260811 01:30:58.167496 7248 consensus_queue.cc:260] T b31aa7bcb4b249bd931573f48ca253de P 4a331de1488e4a0986b490917efea000 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:30:58.168943 7249 tablet_bootstrap.cc:492] T b31aa7bcb4b249bd931573f48ca253de P 10a02c605db44ec69c67a34f53c2a055: No bootstrap required, opened a new log
I20260811 01:30:58.169201 7249 ts_tablet_manager.cc:1397] T b31aa7bcb4b249bd931573f48ca253de P 10a02c605db44ec69c67a34f53c2a055: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.006s
I20260811 01:30:58.169271 6122 catalog_manager.cc:4841] TS 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): tablet 4ac81f2f008042de9813bcf42126d77a (table default.table_c93618b4070e4974a6a794087d73fa73 [id=a65d01fd84fa41a3b31bf35b60f5cbaf]) successfully deleted
I20260811 01:30:58.170265 7249 raft_consensus.cc:348] T b31aa7bcb4b249bd931573f48ca253de P 10a02c605db44ec69c67a34f53c2a055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:30:58.170575 7249 raft_consensus.cc:374] T b31aa7bcb4b249bd931573f48ca253de P 10a02c605db44ec69c67a34f53c2a055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:30:58.170678 7249 raft_consensus.cc:729] T b31aa7bcb4b249bd931573f48ca253de P 10a02c605db44ec69c67a34f53c2a055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10a02c605db44ec69c67a34f53c2a055, State: Initialized, Role: FOLLOWER
I20260811 01:30:58.170997 7249 consensus_queue.cc:260] T b31aa7bcb4b249bd931573f48ca253de P 10a02c605db44ec69c67a34f53c2a055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:30:58.173732 7249 ts_tablet_manager.cc:1428] T b31aa7bcb4b249bd931573f48ca253de P 10a02c605db44ec69c67a34f53c2a055: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 01:30:58.174803 7248 ts_tablet_manager.cc:1428] T b31aa7bcb4b249bd931573f48ca253de P 4a331de1488e4a0986b490917efea000: Time spent starting tablet: real 0.009s user 0.003s sys 0.000s
I20260811 01:30:58.190080 7252 tablet_bootstrap.cc:492] T 8d33739d8c014d8686784505ff5f13ab P 992432fc73c94fcc9f2142b27ef725af: No bootstrap required, opened a new log
I20260811 01:30:58.190413 7252 ts_tablet_manager.cc:1397] T 8d33739d8c014d8686784505ff5f13ab P 992432fc73c94fcc9f2142b27ef725af: Time spent bootstrapping tablet: real 0.052s user 0.019s sys 0.032s
I20260811 01:30:58.191668 7252 raft_consensus.cc:348] T 8d33739d8c014d8686784505ff5f13ab P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:30:58.192083 7252 raft_consensus.cc:374] T 8d33739d8c014d8686784505ff5f13ab P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:30:58.192299 7252 raft_consensus.cc:729] T 8d33739d8c014d8686784505ff5f13ab P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 992432fc73c94fcc9f2142b27ef725af, State: Initialized, Role: FOLLOWER
I20260811 01:30:58.192694 7252 consensus_queue.cc:260] T 8d33739d8c014d8686784505ff5f13ab P 992432fc73c94fcc9f2142b27ef725af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:30:58.193454 7252 ts_tablet_manager.cc:1428] T 8d33739d8c014d8686784505ff5f13ab P 992432fc73c94fcc9f2142b27ef725af: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 01:30:58.194813 7252 tablet_bootstrap.cc:492] T b31aa7bcb4b249bd931573f48ca253de P 992432fc73c94fcc9f2142b27ef725af: Bootstrap starting.
I20260811 01:30:58.196856 7252 tablet_bootstrap.cc:654] T b31aa7bcb4b249bd931573f48ca253de P 992432fc73c94fcc9f2142b27ef725af: Neither blocks nor log segments found. Creating new log.
W20260811 01:30:58.203052 7114 tablet.cc:2369] T 4ac81f2f008042de9813bcf42126d77a P 992432fc73c94fcc9f2142b27ef725af: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:30:58.220552 7252 tablet_bootstrap.cc:492] T b31aa7bcb4b249bd931573f48ca253de P 992432fc73c94fcc9f2142b27ef725af: No bootstrap required, opened a new log
I20260811 01:30:58.220814 7252 ts_tablet_manager.cc:1397] T b31aa7bcb4b249bd931573f48ca253de P 992432fc73c94fcc9f2142b27ef725af: Time spent bootstrapping tablet: real 0.026s user 0.012s sys 0.008s
I20260811 01:30:58.221848 7252 raft_consensus.cc:348] T b31aa7bcb4b249bd931573f48ca253de P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:30:58.222255 7252 raft_consensus.cc:374] T b31aa7bcb4b249bd931573f48ca253de P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:30:58.222390 7252 raft_consensus.cc:729] T b31aa7bcb4b249bd931573f48ca253de P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 992432fc73c94fcc9f2142b27ef725af, State: Initialized, Role: FOLLOWER
I20260811 01:30:58.222764 7252 consensus_queue.cc:260] T b31aa7bcb4b249bd931573f48ca253de P 992432fc73c94fcc9f2142b27ef725af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:30:58.224551 7252 ts_tablet_manager.cc:1428] T b31aa7bcb4b249bd931573f48ca253de P 992432fc73c94fcc9f2142b27ef725af: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
W20260811 01:30:58.282845 6667 tablet.cc:2369] T 8d33739d8c014d8686784505ff5f13ab P 4a331de1488e4a0986b490917efea000: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 01:30:58.286197 7216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:30:58.286733 7216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:30:58.286936 7216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 01:30:58.297317 7260 raft_consensus.cc:482] T b31aa7bcb4b249bd931573f48ca253de P 4a331de1488e4a0986b490917efea000 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260811 01:30:58.297775 7216 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:30:58.297909 7216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:30:58.297986 7216 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
I20260811 01:30:58.297681 7260 raft_consensus.cc:504] T b31aa7bcb4b249bd931573f48ca253de P 4a331de1488e4a0986b490917efea000 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
W20260811 01:30:58.298091 7216 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:30:58.299926 7260 leader_election.cc:288] T b31aa7bcb4b249bd931573f48ca253de P 4a331de1488e4a0986b490917efea000 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433), 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971)
I20260811 01:30:58.312709 6804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b31aa7bcb4b249bd931573f48ca253de" candidate_uuid: "4a331de1488e4a0986b490917efea000" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10a02c605db44ec69c67a34f53c2a055" is_pre_election: true
I20260811 01:30:58.313339 6804 raft_consensus.cc:2454] T b31aa7bcb4b249bd931573f48ca253de P 10a02c605db44ec69c67a34f53c2a055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a331de1488e4a0986b490917efea000 in term 0.
I20260811 01:30:58.315222 7216 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.107.212:36421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:36237
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.107.252:45519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.107.252
--webserver_port=39897
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:30:58.317415 7216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:30:58.320446 7216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 01:30:58.321411 6469 leader_election.cc:302] T b31aa7bcb4b249bd931573f48ca253de P 4a331de1488e4a0986b490917efea000 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000; no voters:
I20260811 01:30:58.323184 7260 raft_consensus.cc:2781] T b31aa7bcb4b249bd931573f48ca253de P 4a331de1488e4a0986b490917efea000 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:30:58.323491 7260 raft_consensus.cc:482] T b31aa7bcb4b249bd931573f48ca253de P 4a331de1488e4a0986b490917efea000 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:30:58.323712 7260 raft_consensus.cc:3037] T b31aa7bcb4b249bd931573f48ca253de P 4a331de1488e4a0986b490917efea000 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:30:58.327926 7260 raft_consensus.cc:504] T b31aa7bcb4b249bd931573f48ca253de P 4a331de1488e4a0986b490917efea000 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:30:58.330308 6804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b31aa7bcb4b249bd931573f48ca253de" candidate_uuid: "4a331de1488e4a0986b490917efea000" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10a02c605db44ec69c67a34f53c2a055"
I20260811 01:30:58.330652 6804 raft_consensus.cc:3037] T b31aa7bcb4b249bd931573f48ca253de P 10a02c605db44ec69c67a34f53c2a055 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:30:58.336012 6804 raft_consensus.cc:2454] T b31aa7bcb4b249bd931573f48ca253de P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a331de1488e4a0986b490917efea000 in term 1.
I20260811 01:30:58.336647 7260 leader_election.cc:288] T b31aa7bcb4b249bd931573f48ca253de P 4a331de1488e4a0986b490917efea000 [CANDIDATE]: Term 1 election: Requested vote from peers 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433), 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971)
I20260811 01:30:58.336815 6469 leader_election.cc:302] T b31aa7bcb4b249bd931573f48ca253de P 4a331de1488e4a0986b490917efea000 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000; no voters:
I20260811 01:30:58.337324 7260 raft_consensus.cc:2781] T b31aa7bcb4b249bd931573f48ca253de P 4a331de1488e4a0986b490917efea000 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:30:58.348451 7260 raft_consensus.cc:686] T b31aa7bcb4b249bd931573f48ca253de P 4a331de1488e4a0986b490917efea000 [term 1 LEADER]: Becoming Leader. State: Replica: 4a331de1488e4a0986b490917efea000, State: Running, Role: LEADER
I20260811 01:30:58.349087 7260 consensus_queue.cc:237] T b31aa7bcb4b249bd931573f48ca253de P 4a331de1488e4a0986b490917efea000 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:30:58.352138 7040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b31aa7bcb4b249bd931573f48ca253de" candidate_uuid: "4a331de1488e4a0986b490917efea000" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "992432fc73c94fcc9f2142b27ef725af" is_pre_election: true
I20260811 01:30:58.352850 7040 raft_consensus.cc:2454] T b31aa7bcb4b249bd931573f48ca253de P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a331de1488e4a0986b490917efea000 in term 0.
I20260811 01:30:58.353538 7036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b31aa7bcb4b249bd931573f48ca253de" candidate_uuid: "4a331de1488e4a0986b490917efea000" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "992432fc73c94fcc9f2142b27ef725af"
I20260811 01:30:58.353834 7036 raft_consensus.cc:3037] T b31aa7bcb4b249bd931573f48ca253de P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:30:58.357182 7036 raft_consensus.cc:2454] T b31aa7bcb4b249bd931573f48ca253de P 992432fc73c94fcc9f2142b27ef725af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a331de1488e4a0986b490917efea000 in term 1.
I20260811 01:30:58.364243 7260 raft_consensus.cc:482] T 8d33739d8c014d8686784505ff5f13ab P 4a331de1488e4a0986b490917efea000 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:30:58.364485 7260 raft_consensus.cc:504] T 8d33739d8c014d8686784505ff5f13ab P 4a331de1488e4a0986b490917efea000 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:30:58.365339 7260 leader_election.cc:288] T 8d33739d8c014d8686784505ff5f13ab P 4a331de1488e4a0986b490917efea000 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:30:58.365262 7295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:30:58.366958 6804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d33739d8c014d8686784505ff5f13ab" candidate_uuid: "4a331de1488e4a0986b490917efea000" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10a02c605db44ec69c67a34f53c2a055" is_pre_election: true
I20260811 01:30:58.367269 6804 raft_consensus.cc:2454] T 8d33739d8c014d8686784505ff5f13ab P 10a02c605db44ec69c67a34f53c2a055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a331de1488e4a0986b490917efea000 in term 0.
I20260811 01:30:58.367686 7036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d33739d8c014d8686784505ff5f13ab" candidate_uuid: "4a331de1488e4a0986b490917efea000" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "992432fc73c94fcc9f2142b27ef725af" is_pre_election: true
I20260811 01:30:58.367990 7036 raft_consensus.cc:2454] T 8d33739d8c014d8686784505ff5f13ab P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a331de1488e4a0986b490917efea000 in term 0.
I20260811 01:30:58.368767 6470 leader_election.cc:302] T 8d33739d8c014d8686784505ff5f13ab P 4a331de1488e4a0986b490917efea000 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af; no voters:
I20260811 01:30:58.369606 7260 raft_consensus.cc:2781] T 8d33739d8c014d8686784505ff5f13ab P 4a331de1488e4a0986b490917efea000 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:30:58.369712 7260 raft_consensus.cc:482] T 8d33739d8c014d8686784505ff5f13ab P 4a331de1488e4a0986b490917efea000 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:30:58.369813 7260 raft_consensus.cc:3037] T 8d33739d8c014d8686784505ff5f13ab P 4a331de1488e4a0986b490917efea000 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:30:58.372864 7260 raft_consensus.cc:504] T 8d33739d8c014d8686784505ff5f13ab P 4a331de1488e4a0986b490917efea000 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:30:58.374281 6804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d33739d8c014d8686784505ff5f13ab" candidate_uuid: "4a331de1488e4a0986b490917efea000" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10a02c605db44ec69c67a34f53c2a055"
I20260811 01:30:58.374562 6804 raft_consensus.cc:3037] T 8d33739d8c014d8686784505ff5f13ab P 10a02c605db44ec69c67a34f53c2a055 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:30:58.378042 6804 raft_consensus.cc:2454] T 8d33739d8c014d8686784505ff5f13ab P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a331de1488e4a0986b490917efea000 in term 1.
I20260811 01:30:58.379114 6469 leader_election.cc:302] T 8d33739d8c014d8686784505ff5f13ab P 4a331de1488e4a0986b490917efea000 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000; no voters:
I20260811 01:30:58.379536 7290 raft_consensus.cc:2781] T 8d33739d8c014d8686784505ff5f13ab P 4a331de1488e4a0986b490917efea000 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:30:58.382807 7036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d33739d8c014d8686784505ff5f13ab" candidate_uuid: "4a331de1488e4a0986b490917efea000" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "992432fc73c94fcc9f2142b27ef725af"
I20260811 01:30:58.383145 7036 raft_consensus.cc:3037] T 8d33739d8c014d8686784505ff5f13ab P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:30:58.388255 7036 raft_consensus.cc:2454] T 8d33739d8c014d8686784505ff5f13ab P 992432fc73c94fcc9f2142b27ef725af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a331de1488e4a0986b490917efea000 in term 1.
I20260811 01:30:58.389194 7290 raft_consensus.cc:686] T 8d33739d8c014d8686784505ff5f13ab P 4a331de1488e4a0986b490917efea000 [term 1 LEADER]: Becoming Leader. State: Replica: 4a331de1488e4a0986b490917efea000, State: Running, Role: LEADER
I20260811 01:30:58.389621 7290 consensus_queue.cc:237] T 8d33739d8c014d8686784505ff5f13ab P 4a331de1488e4a0986b490917efea000 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:30:58.391250 7260 leader_election.cc:288] T 8d33739d8c014d8686784505ff5f13ab P 4a331de1488e4a0986b490917efea000 [CANDIDATE]: Term 1 election: Requested vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:30:58.364045 7297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:30:58.402089 7216 server_base.cc:1034] running on GCE node
W20260811 01:30:58.402411 7300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
01:30:58.402 [HMS - ERROR - pool-6-thread-3] (RetryingHMSHandler.java:201) AlreadyExistsException(message:Table hive.default.table_d472cfe99d004f68bc6beda9b7c20bc4 already exists)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1839)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1786)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_with_environment_context(HiveMetaStore.java:2035)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
at com.sun.proxy.$Proxy25.create_table_with_environment_context(Unknown Source)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:14800)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:14784)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:104)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
I20260811 01:30:58.403927 7216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:30:58.406563 7216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
W20260811 01:30:58.406538 6135 catalog_manager.cc:2144] Already present: failed to create HMS catalog entry for table [id=95f6ffb714d7427e9beda3cc437c1539]: failed to create Hive MetaStore table: TException - service has thrown: AlreadyExistsException(message=Table hive.default.table_d472cfe99d004f68bc6beda9b7c20bc4 already exists)
I20260811 01:30:58.408008 7216 hybrid_clock.cc:648] HybridClock initialized: now 1786411858407857 us; error 104 us; skew 500 ppm
I20260811 01:30:58.408543 7216 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:30:58.409605 7009 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ac81f2f008042de9813bcf42126d77a with delete_type TABLET_DATA_DELETED (replaced by tablet b31aa7bcb4b249bd931573f48ca253de) from {username='slave'} at 127.0.0.1:39622
I20260811 01:30:58.409977 7268 tablet_replica.cc:330] T 4ac81f2f008042de9813bcf42126d77a P 992432fc73c94fcc9f2142b27ef725af: stopping tablet replica
I20260811 01:30:58.410530 7268 raft_consensus.cc:2229] T 4ac81f2f008042de9813bcf42126d77a P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 01:30:58.411017 7268 raft_consensus.cc:2258] T 4ac81f2f008042de9813bcf42126d77a P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 01:30:58.412868 7268 ts_tablet_manager.cc:1905] T 4ac81f2f008042de9813bcf42126d77a P 992432fc73c94fcc9f2142b27ef725af: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 01:30:58.414646 6133 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_071a47d589c9407aa47c9d668c44cc23 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 01:30:58.414655 6130 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_b6731279f2fb4ef2a0a845c55840c58b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 01:30:58.414944 6140 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_977e759aa4644f0091ecc8c4613cffc2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 01:30:58.415231 6142 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_1354313d091b46d88f34c2ccd44bf8e0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 01:30:58.416023 6141 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_34b4086ff0f64632adbac74648d9874a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 01:30:58.416188 6146 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_4fad8fd88a6444b2a3dbc99d87835413 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 01:30:58.416828 6131 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_16925b327c134e08987029291b57034c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 01:30:58.419525 7268 ts_tablet_manager.cc:1918] T 4ac81f2f008042de9813bcf42126d77a P 992432fc73c94fcc9f2142b27ef725af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 01:30:58.419726 7268 log.cc:1196] T 4ac81f2f008042de9813bcf42126d77a P 992432fc73c94fcc9f2142b27ef725af: Deleting WAL directory at /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-2/wal/wals/4ac81f2f008042de9813bcf42126d77a
I20260811 01:30:58.420162 7268 ts_tablet_manager.cc:1939] T 4ac81f2f008042de9813bcf42126d77a P 992432fc73c94fcc9f2142b27ef725af: Deleting consensus metadata
I20260811 01:30:58.421933 6124 catalog_manager.cc:4841] TS 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): tablet 4ac81f2f008042de9813bcf42126d77a (table default.table_c93618b4070e4974a6a794087d73fa73 [id=a65d01fd84fa41a3b31bf35b60f5cbaf]) successfully deleted
I20260811 01:30:58.426278 6135 catalog_manager.cc:5495] T 8d33739d8c014d8686784505ff5f13ab P 4a331de1488e4a0986b490917efea000 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a331de1488e4a0986b490917efea000 (127.5.107.193). New cstate: current_term: 1 leader_uuid: "4a331de1488e4a0986b490917efea000" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } health_report { overall_health: UNKNOWN } } }
I20260811 01:30:58.427356 6135 catalog_manager.cc:5495] T b31aa7bcb4b249bd931573f48ca253de P 4a331de1488e4a0986b490917efea000 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a331de1488e4a0986b490917efea000 (127.5.107.193). New cstate: current_term: 1 leader_uuid: "4a331de1488e4a0986b490917efea000" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } health_report { overall_health: UNKNOWN } } }
F20260811 01:30:58.432898 6442 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_d472cfe99d004f68bc6beda9b7c20bc4
*** Check failure stack trace: ***
@ 0x7f2df6d329ad google::LogMessage::Fail() at ??:0
@ 0x7f2df6d3588a google::LogMessage::SendToLog() at ??:0
I20260811 01:30:58.439612 6135 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34732:
name: "default.Table_e6b7b1e823f84e859b8956dc8c474b29"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
@ 0x7f2df6d324de google::LogMessage::Flush() at ??:0
@ 0x7f2df6d326d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7f2e0c8d0ba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260811 01:30:58.466259 7216 webserver.cc:460] Webserver started at http://127.5.107.252:39897/ using document root <none> and password file <none>
I20260811 01:30:58.467499 7216 fs_manager.cc:359] Metadata directory not provided
I20260811 01:30:58.467772 7216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
@ 0x7f2e0c8cf901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f2e0c8d666c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7f2e0c8d64e9 std::_Function_handler<>::_M_invoke() at ??:0
I20260811 01:30:58.488658 7216 fs_manager.cc:709] Time spent opening directory manager: real 0.018s user 0.010s sys 0.001s
@ 0x9edb7d std::function<>::operator()() at ??:?
I20260811 01:30:58.489739 6130 catalog_manager.cc:2148] created HMS catalog entry for table [id=89b85d47429d40c787b453da0787febc]
I20260811 01:30:58.495296 7319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:30:58.497453 6130 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:34732 (request call id 39) took 1049 ms. Trace:
I20260811 01:30:58.497646 6130 rpcz_store.cc:276] 0811 01:30:57.447713 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 01:30:57.447781 (+ 68us) service_pool.cc:227] Handling call
0811 01:30:58.414846 (+967065us) catalog_manager.cc:2075] Acquired catalog manager lock
0811 01:30:58.415517 (+ 671us) catalog_manager.cc:2121] Created new table and tablet info
0811 01:30:58.489654 (+ 74137us) catalog_manager.cc:2147] Created new table in HMS catalog
0811 01:30:58.497112 (+ 7458us) catalog_manager.cc:2181] Wrote table and tablets to system table
0811 01:30:58.497273 (+ 161us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0811 01:30:58.497398 (+ 125us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 01:30:58.490763 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 01:30:58.490913 (+ 150us) write_op.cc:432] Acquiring schema lock in shared mode
0811 01:30:58.490920 (+ 7us) write_op.cc:435] Acquired schema lock
0811 01:30:58.490922 (+ 2us) tablet.cc:660] Decoding operations
0811 01:30:58.491044 (+ 122us) write_op.cc:620] Acquiring the partition lock for write op
0811 01:30:58.491059 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0811 01:30:58.491064 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
0811 01:30:58.491155 (+ 91us) tablet.cc:699] Row locks acquired
0811 01:30:58.491157 (+ 2us) write_op.cc:260] PREPARE: finished
0811 01:30:58.491203 (+ 46us) write_op.cc:270] Start()
0811 01:30:58.491244 (+ 41us) write_op.cc:276] Timestamp: P: 1786411858491168 usec, L: 0
0811 01:30:58.491248 (+ 4us) op_driver.cc:331] REPLICATION: starting
0811 01:30:58.491489 (+ 241us) log.cc:842] Serialized 432 byte log entry
0811 01:30:58.496276 (+ 4787us) op_driver.cc:444] REPLICATION: finished
0811 01:30:58.496441 (+ 165us) write_op.cc:301] APPLY: starting
0811 01:30:58.496464 (+ 23us) tablet.cc:1365] starting BulkCheckPresence
0811 01:30:58.496526 (+ 62us) tablet.cc:1368] finished BulkCheckPresence
0811 01:30:58.496575 (+ 49us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 01:30:58.496710 (+ 135us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 01:30:58.496759 (+ 49us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0811 01:30:58.496768 (+ 9us) write_op.cc:312] APPLY: finished
0811 01:30:58.496865 (+ 97us) log.cc:842] Serialized 28 byte log entry
0811 01:30:58.496967 (+ 102us) write_op.cc:489] Releasing partition, row and schema locks
0811 01:30:58.496994 (+ 27us) write_op.cc:454] Released schema lock
0811 01:30:58.497003 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0811 01:30:58.497079 (+ 76us) write_op.cc:489] Releasing partition, row and schema locks
0811 01:30:58.497085 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":17002,"Hive Metastore.run_cpu_time_us":357,"Hive Metastore.run_wall_time_us":56914,"spinlock_wait_cycles":6784,"child_traces":[["op",{"apply.queue_time_us":101,"apply.run_cpu_time_us":706,"apply.run_wall_time_us":705,"num_ops":2,"prepare.queue_time_us":58,"prepare.run_cpu_time_us":932,"prepare.run_wall_time_us":932,"raft.queue_time_us":1070,"raft.run_cpu_time_us":293,"raft.run_wall_time_us":293,"replication_time_us":4997}]]}
I20260811 01:30:58.499755 6130 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34732:
name: "default.Table_9f9bb4e0b22447c79a5597bc7db4eec8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:30:58.500543 7216 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.005s sys 0.000s
I20260811 01:30:58.500758 7216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/data,/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/wal
uuid: "c6f417c29b464762944a721d477b2b13"
format_stamp: "Formatted at 2026-08-11 01:30:54 on dist-test-slave-2p8l"
@ 0x7f2df995ceaf kudu::Thread::SuperviseThread() at ??:0
I20260811 01:30:58.502351 7216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
@ 0x7f2dfcc366db start_thread at ??:0
@ 0x7f2df5bcc61f clone at ??:0
I20260811 01:30:58.551589 7216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:30:58.553139 7216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:30:58.553618 7216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:30:58.591293 7216 rpc_server.cc:307] RPC server started. Bound to: 127.5.107.252:45519
I20260811 01:30:58.591305 7428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.107.252:45519 every 8 connection(s)
I20260811 01:30:58.593639 7216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/data/info.pb
I20260811 01:30:58.600405 5551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZLXb2N/build/asan/bin/kudu as pid 7216
I20260811 01:30:58.600881 7429 sys_catalog.cc:263] Verifying existing consensus state
I20260811 01:30:58.613646 7429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13: Bootstrap starting.
I20260811 01:30:58.623467 7429 log.cc:824] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13: Log is configured to *not* fsync() on all Append() calls
I20260811 01:30:58.635497 7429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 01:30:58.636317 7429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13: Bootstrap complete.
I20260811 01:30:58.648581 7429 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:30:58.649986 7429 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6f417c29b464762944a721d477b2b13, State: Initialized, Role: FOLLOWER
I20260811 01:30:58.650780 7429 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:30:58.651660 7437 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } } }
I20260811 01:30:58.652045 7437 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:30:58.653187 7429 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:30:58.673295 7466 catalog_manager.cc:1259] Loaded cluster ID: 2f0039b0f9ec4f479f59cf7d8b74b7a2
I20260811 01:30:58.673425 7466 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13: loading cluster ID for follower catalog manager: success
I20260811 01:30:58.690255 7466 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13: acquiring CA information for follower catalog manager: success
I20260811 01:30:58.695291 7466 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 01:30:58.698496 6890 heartbeater.cc:347] Connected to a master server at 127.5.107.252:45519
I20260811 01:30:58.702603 7368 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" instance_seqno: 1786411855798958) as {username='slave'} at 127.5.107.194:38941; Asking this server to re-register.
I20260811 01:30:58.703616 6890 heartbeater.cc:464] Registering TS with master...
I20260811 01:30:58.705629 7368 ts_manager.cc:194] Registered new tserver with Master: 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971)
I20260811 01:30:58.739478 7110 heartbeater.cc:347] Connected to a master server at 127.5.107.252:45519
I20260811 01:30:58.739759 7110 heartbeater.cc:503] Master 127.5.107.252:45519 was elected leader, sending a full tablet report...
I20260811 01:30:58.741508 7368 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" instance_seqno: 1786411856431448) as {username='slave'} at 127.5.107.195:57559; Asking this server to re-register.
I20260811 01:30:58.742321 7110 heartbeater.cc:464] Registering TS with master...
I20260811 01:30:58.743883 7368 ts_manager.cc:194] Registered new tserver with Master: 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:30:58.789414 7131 connection.cc:535] client connection to 127.5.107.253:36427 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 01:30:58.790439 7131 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.5.107.253:36427)
W20260811 01:30:58.790874 7131 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.5.107.253:36427)
W20260811 01:30:58.791399 7131 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.5.107.253:36427)
W20260811 01:30:58.792361 7470 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:30:58.692537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.107.253:36427 (local address 127.0.0.1:34742)
0811 01:30:58.692902 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0811 01:30:58.692922 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 01:30:58.693032 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:30:58.790911 (+ 97879us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":237}
I20260811 01:30:58.793879 7216 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107),
I20260811 01:30:58.805305 7216 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:30:58.896270 6654 heartbeater.cc:347] Connected to a master server at 127.5.107.252:45519
I20260811 01:30:58.898243 7368 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "4a331de1488e4a0986b490917efea000" instance_seqno: 1786411855240795) as {username='slave'} at 127.5.107.193:45833; Asking this server to re-register.
I20260811 01:30:58.898945 6654 heartbeater.cc:464] Registering TS with master...
I20260811 01:30:58.900388 7368 ts_manager.cc:194] Registered new tserver with Master: 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
I20260811 01:30:59.006323 7480 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8970a89e5fd744008240ecd668272a4a)
I20260811 01:30:59.006605 7480 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:30:59.007509 7480 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:30:59.009225 6004 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111) [suppressed 9 similar messages]
W20260811 01:30:59.011904 6004 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
I20260811 01:30:59.017381 7390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "61c27cf3d4c846748ae6752e0fa652b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "c6f417c29b464762944a721d477b2b13" is_pre_election: true
I20260811 01:30:59.018385 7390 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61c27cf3d4c846748ae6752e0fa652b9 in term 1.
I20260811 01:30:59.019177 6004 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9, c6f417c29b464762944a721d477b2b13; no voters: 8970a89e5fd744008240ecd668272a4a
I20260811 01:30:59.019497 7480 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260811 01:30:59.019654 7480 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8970a89e5fd744008240ecd668272a4a)
I20260811 01:30:59.019767 7480 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 2 FOLLOWER]: Advancing to term 3
I20260811 01:30:59.021986 7480 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:30:59.022708 7480 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 3 election: Requested vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
I20260811 01:30:59.023593 7390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "61c27cf3d4c846748ae6752e0fa652b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "c6f417c29b464762944a721d477b2b13"
I20260811 01:30:59.023861 7390 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 1 FOLLOWER]: Advancing to term 3
W20260811 01:30:59.024498 6004 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
I20260811 01:30:59.029320 7390 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61c27cf3d4c846748ae6752e0fa652b9 in term 3.
I20260811 01:30:59.029922 6004 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9, c6f417c29b464762944a721d477b2b13; no voters: 8970a89e5fd744008240ecd668272a4a
I20260811 01:30:59.030372 7480 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 01:30:59.031008 7480 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 3 LEADER]: Becoming Leader. State: Replica: 61c27cf3d4c846748ae6752e0fa652b9, State: Running, Role: LEADER
I20260811 01:30:59.031509 7480 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:30:59.033113 7489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61c27cf3d4c846748ae6752e0fa652b9. Latest consensus state: current_term: 3 leader_uuid: "61c27cf3d4c846748ae6752e0fa652b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } } }
I20260811 01:30:59.033560 7489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [sys.catalog]: This master's current role is: LEADER
I20260811 01:30:59.219213 7494 consensus_queue.cc:1035] T b31aa7bcb4b249bd931573f48ca253de P 4a331de1488e4a0986b490917efea000 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:30:59.228611 7498 consensus_queue.cc:1035] T b31aa7bcb4b249bd931573f48ca253de P 4a331de1488e4a0986b490917efea000 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:30:59.267894 7498 consensus_queue.cc:1035] T 8d33739d8c014d8686784505ff5f13ab P 4a331de1488e4a0986b490917efea000 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:30:59.273593 7501 consensus_queue.cc:1035] T 8d33739d8c014d8686784505ff5f13ab P 4a331de1488e4a0986b490917efea000 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 01:30:59.540548 6004 consensus_peers.cc:487] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 -> Peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Couldn't send request to peer 8970a89e5fd744008240ecd668272a4a. Status: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 01:30:59.639786 7390 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Refusing update from remote peer 61c27cf3d4c846748ae6752e0fa652b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 15. (index mismatch)
I20260811 01:30:59.640672 7513 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 4, Time since last communication: 0.000s
I20260811 01:30:59.641465 5976 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:30:59.641659 5976 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:30:59.648356 7523 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "61c27cf3d4c846748ae6752e0fa652b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } } }
I20260811 01:30:59.648837 7523 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:30:59.649329 7518 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61c27cf3d4c846748ae6752e0fa652b9. Latest consensus state: current_term: 3 leader_uuid: "61c27cf3d4c846748ae6752e0fa652b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } } }
I20260811 01:30:59.649734 7518 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:30:59.651747 7526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "61c27cf3d4c846748ae6752e0fa652b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } } }
I20260811 01:30:59.652089 7526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [sys.catalog]: This master's current role is: LEADER
I20260811 01:30:59.653661 7519 mvcc.cc:205] Tried to move back new op lower bound from 7317142972379824128 to 7317142967951904768. Current Snapshot: MvccSnapshot[applied={T|T < 7317142972379824128 or (T in {7317142972379824128})}]
I20260811 01:30:59.654450 7518 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "61c27cf3d4c846748ae6752e0fa652b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } } }
I20260811 01:30:59.654738 7518 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:30:59.679520 7491 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:30:59.682615 7491 catalog_manager.cc:669] Loaded metadata for table default.table_f7f5b09418c242d092b1948b866994e5 [id=69f9a5a340894b4bb41b150af86e0d8a]
I20260811 01:30:59.683226 7491 catalog_manager.cc:669] Loaded metadata for table default.table_b6731279f2fb4ef2a0a845c55840c58b [id=89b85d47429d40c787b453da0787febc]
I20260811 01:30:59.683624 7491 catalog_manager.cc:669] Loaded metadata for table default.table_c93618b4070e4974a6a794087d73fa73 [id=a65d01fd84fa41a3b31bf35b60f5cbaf]
I20260811 01:30:59.685593 7491 tablet_loader.cc:96] loaded metadata for tablet 24df17b4b73c4838aa887fe2c1d4bcd0 (table default.table_b6731279f2fb4ef2a0a845c55840c58b [id=89b85d47429d40c787b453da0787febc])
I20260811 01:30:59.686241 7491 tablet_loader.cc:96] loaded metadata for tablet 8d33739d8c014d8686784505ff5f13ab (table default.table_f7f5b09418c242d092b1948b866994e5 [id=69f9a5a340894b4bb41b150af86e0d8a])
I20260811 01:30:59.686589 7491 tablet_loader.cc:96] loaded metadata for tablet b31aa7bcb4b249bd931573f48ca253de (table default.table_c93618b4070e4974a6a794087d73fa73 [id=a65d01fd84fa41a3b31bf35b60f5cbaf])
I20260811 01:30:59.686858 7491 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:30:59.687820 7491 catalog_manager.cc:1259] Loaded cluster ID: 2f0039b0f9ec4f479f59cf7d8b74b7a2
I20260811 01:30:59.687940 7491 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:30:59.689231 7491 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:30:59.690222 7491 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9: Loaded TSK: 0
I20260811 01:30:59.691136 7491 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 01:30:59.692049 7491 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260811 01:30:59.692914 7491 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260811 01:30:59.725018 6023 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46594:
table { table_name: "default.Table_b6731279f2fb4ef2a0a845c55840c58b" } new_table_name: "default.Table_ae0986f440b7487789e0721a121aa6d3" modify_external_catalogs: true
I20260811 01:30:59.725054 6025 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46594:
name: "default.Table_5e80cfaf385a44328ab0113196274005"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:30:59.725543 6022 master_service.cc:946] ReplaceTablet: received request to replace tablet b31aa7bcb4b249bd931573f48ca253de from {username='slave'} at 127.0.0.1:46594
I20260811 01:30:59.725037 6024 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46594:
name: "default.Table_0bf1f8de8ab54285b6291b6c6e30b647"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:30:59.726410 6021 master_service.cc:946] ReplaceTablet: received request to replace tablet 8d33739d8c014d8686784505ff5f13ab from {username='slave'} at 127.0.0.1:46594
I20260811 01:30:59.726850 6020 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46594:
name: "default.Table_977e759aa4644f0091ecc8c4613cffc2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:30:59.728550 6017 master_service.cc:946] ReplaceTablet: received request to replace tablet 8d33739d8c014d8686784505ff5f13ab from {username='slave'} at 127.0.0.1:46594
I20260811 01:30:59.728924 6014 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46594:
name: "default.Table_34b4086ff0f64632adbac74648d9874a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:30:59.729628 6013 master_service.cc:946] ReplaceTablet: received request to replace tablet b31aa7bcb4b249bd931573f48ca253de from {username='slave'} at 127.0.0.1:46594
I20260811 01:30:59.731000 6011 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46594:
name: "default.Table_1354313d091b46d88f34c2ccd44bf8e0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260811 01:30:59.732705 6417 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_d472cfe99d004f68bc6beda9b7c20bc4
*** Check failure stack trace: ***
@ 0x7fddbecb19ad google::LogMessage::Fail() at ??:0
I20260811 01:30:59.738521 6021 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9: Sending DeleteTablet for 3 replicas of tablet 8d33739d8c014d8686784505ff5f13ab
I20260811 01:30:59.740512 6017 master_service.cc:946] ReplaceTablet: received request to replace tablet 8d33739d8c014d8686784505ff5f13ab from {username='slave'} at 127.0.0.1:46594
@ 0x7fddbecb488a google::LogMessage::SendToLog() at ??:0
@ 0x7fddbecb14de google::LogMessage::Flush() at ??:0
@ 0x7fddbecb16d1 google::LogMessage::~LogMessage() at ??:0
I20260811 01:30:59.756524 6021 catalog_manager.cc:6565] ReplaceTablet: tablet 8d33739d8c014d8686784505ff5f13ab deleted and replaced by tablet a6bc3c3d5d904830bfda3966e47305da
I20260811 01:30:59.763146 6557 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d33739d8c014d8686784505ff5f13ab with delete_type TABLET_DATA_DELETED (replaced by tablet a6bc3c3d5d904830bfda3966e47305da) from {username='slave'} at 127.0.0.1:49336
I20260811 01:30:59.767431 6022 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9: Sending DeleteTablet for 3 replicas of tablet b31aa7bcb4b249bd931573f48ca253de
I20260811 01:30:59.767980 6022 catalog_manager.cc:6565] ReplaceTablet: tablet b31aa7bcb4b249bd931573f48ca253de deleted and replaced by tablet 1cb26d431b24475a8cf62325267b0a3d
I20260811 01:30:59.768503 7550 tablet_replica.cc:330] T 8d33739d8c014d8686784505ff5f13ab P 4a331de1488e4a0986b490917efea000: stopping tablet replica
I20260811 01:30:59.768429 6022 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46594:
name: "default.Table_e6b7b1e823f84e859b8956dc8c474b29"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:30:59.768760 7550 raft_consensus.cc:2229] T 8d33739d8c014d8686784505ff5f13ab P 4a331de1488e4a0986b490917efea000 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:30:59.769476 7550 raft_consensus.cc:2258] T 8d33739d8c014d8686784505ff5f13ab P 4a331de1488e4a0986b490917efea000 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:30:59.769438 6013 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46594:
name: "default.Table_071a47d589c9407aa47c9d668c44cc23"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:30:59.770619 6017 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46594:
name: "default.Table_16925b327c134e08987029291b57034c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:30:59.771144 6557 tablet_service.cc:1514] Processing DeleteTablet for tablet b31aa7bcb4b249bd931573f48ca253de with delete_type TABLET_DATA_DELETED (replaced by tablet 1cb26d431b24475a8cf62325267b0a3d) from {username='slave'} at 127.0.0.1:49336
I20260811 01:30:59.772410 7550 ts_tablet_manager.cc:1905] T 8d33739d8c014d8686784505ff5f13ab P 4a331de1488e4a0986b490917efea000: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:30:59.777397 6021 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46594:
name: "default.Table_4fad8fd88a6444b2a3dbc99d87835413"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
@ 0x7fddd484fba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260811 01:30:59.787343 6557 tablet_service.cc:1467] Processing CreateTablet for tablet a6bc3c3d5d904830bfda3966e47305da (DEFAULT_TABLE table=default.table_f7f5b09418c242d092b1948b866994e5 [id=69f9a5a340894b4bb41b150af86e0d8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:30:59.787736 7550 ts_tablet_manager.cc:1918] T 8d33739d8c014d8686784505ff5f13ab P 4a331de1488e4a0986b490917efea000: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 01:30:59.787875 6557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6bc3c3d5d904830bfda3966e47305da. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7fddd484e901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260811 01:30:59.795018 6554 tablet_service.cc:1467] Processing CreateTablet for tablet 1cb26d431b24475a8cf62325267b0a3d (DEFAULT_TABLE table=default.table_c93618b4070e4974a6a794087d73fa73 [id=a65d01fd84fa41a3b31bf35b60f5cbaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:30:59.795601 6554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1cb26d431b24475a8cf62325267b0a3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:30:59.799187 6770 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d33739d8c014d8686784505ff5f13ab with delete_type TABLET_DATA_DELETED (replaced by tablet a6bc3c3d5d904830bfda3966e47305da) from {username='slave'} at 127.0.0.1:41458
I20260811 01:30:59.799592 6774 tablet_service.cc:1467] Processing CreateTablet for tablet a6bc3c3d5d904830bfda3966e47305da (DEFAULT_TABLE table=default.table_f7f5b09418c242d092b1948b866994e5 [id=69f9a5a340894b4bb41b150af86e0d8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:30:59.787894 7550 log.cc:1196] T 8d33739d8c014d8686784505ff5f13ab P 4a331de1488e4a0986b490917efea000: Deleting WAL directory at /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-0/wal/wals/8d33739d8c014d8686784505ff5f13ab
I20260811 01:30:59.799722 6773 tablet_service.cc:1514] Processing DeleteTablet for tablet b31aa7bcb4b249bd931573f48ca253de with delete_type TABLET_DATA_DELETED (replaced by tablet 1cb26d431b24475a8cf62325267b0a3d) from {username='slave'} at 127.0.0.1:41458
I20260811 01:30:59.800145 6774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6bc3c3d5d904830bfda3966e47305da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:30:59.800351 7550 ts_tablet_manager.cc:1939] T 8d33739d8c014d8686784505ff5f13ab P 4a331de1488e4a0986b490917efea000: Deleting consensus metadata
I20260811 01:30:59.800335 6769 tablet_service.cc:1467] Processing CreateTablet for tablet 1cb26d431b24475a8cf62325267b0a3d (DEFAULT_TABLE table=default.table_c93618b4070e4974a6a794087d73fa73 [id=a65d01fd84fa41a3b31bf35b60f5cbaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:30:59.800925 6769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1cb26d431b24475a8cf62325267b0a3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:30:59.801169 7557 tablet_replica.cc:330] T 8d33739d8c014d8686784505ff5f13ab P 10a02c605db44ec69c67a34f53c2a055: stopping tablet replica
I20260811 01:30:59.801388 7557 raft_consensus.cc:2229] T 8d33739d8c014d8686784505ff5f13ab P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:30:59.801568 7550 tablet_replica.cc:330] T b31aa7bcb4b249bd931573f48ca253de P 4a331de1488e4a0986b490917efea000: stopping tablet replica
I20260811 01:30:59.801734 7557 raft_consensus.cc:2258] T 8d33739d8c014d8686784505ff5f13ab P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:30:59.801775 7550 raft_consensus.cc:2229] T b31aa7bcb4b249bd931573f48ca253de P 4a331de1488e4a0986b490917efea000 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:30:59.802165 7550 raft_consensus.cc:2258] T b31aa7bcb4b249bd931573f48ca253de P 4a331de1488e4a0986b490917efea000 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:30:59.803047 7550 ts_tablet_manager.cc:1905] T b31aa7bcb4b249bd931573f48ca253de P 4a331de1488e4a0986b490917efea000: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:30:59.803125 7557 ts_tablet_manager.cc:1905] T 8d33739d8c014d8686784505ff5f13ab P 10a02c605db44ec69c67a34f53c2a055: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:30:59.807495 7564 tablet_bootstrap.cc:492] T a6bc3c3d5d904830bfda3966e47305da P 4a331de1488e4a0986b490917efea000: Bootstrap starting.
I20260811 01:30:59.813761 7550 ts_tablet_manager.cc:1918] T b31aa7bcb4b249bd931573f48ca253de P 4a331de1488e4a0986b490917efea000: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 01:30:59.813957 7550 log.cc:1196] T b31aa7bcb4b249bd931573f48ca253de P 4a331de1488e4a0986b490917efea000: Deleting WAL directory at /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-0/wal/wals/b31aa7bcb4b249bd931573f48ca253de
I20260811 01:30:59.814482 7550 ts_tablet_manager.cc:1939] T b31aa7bcb4b249bd931573f48ca253de P 4a331de1488e4a0986b490917efea000: Deleting consensus metadata
I20260811 01:30:59.821194 7557 ts_tablet_manager.cc:1918] T 8d33739d8c014d8686784505ff5f13ab P 10a02c605db44ec69c67a34f53c2a055: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 01:30:59.821372 7557 log.cc:1196] T 8d33739d8c014d8686784505ff5f13ab P 10a02c605db44ec69c67a34f53c2a055: Deleting WAL directory at /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-1/wal/wals/8d33739d8c014d8686784505ff5f13ab
I20260811 01:30:59.821914 7557 ts_tablet_manager.cc:1939] T 8d33739d8c014d8686784505ff5f13ab P 10a02c605db44ec69c67a34f53c2a055: Deleting consensus metadata
I20260811 01:30:59.827196 7564 tablet_bootstrap.cc:654] T a6bc3c3d5d904830bfda3966e47305da P 4a331de1488e4a0986b490917efea000: Neither blocks nor log segments found. Creating new log.
I20260811 01:30:59.831022 7557 tablet_replica.cc:330] T b31aa7bcb4b249bd931573f48ca253de P 10a02c605db44ec69c67a34f53c2a055: stopping tablet replica
I20260811 01:30:59.831290 7557 raft_consensus.cc:2229] T b31aa7bcb4b249bd931573f48ca253de P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:30:59.831512 7557 raft_consensus.cc:2258] T b31aa7bcb4b249bd931573f48ca253de P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:30:59.832917 5999 catalog_manager.cc:4841] TS 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): tablet 8d33739d8c014d8686784505ff5f13ab (table default.table_f7f5b09418c242d092b1948b866994e5 [id=69f9a5a340894b4bb41b150af86e0d8a]) successfully deleted
I20260811 01:30:59.833117 7009 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d33739d8c014d8686784505ff5f13ab with delete_type TABLET_DATA_DELETED (replaced by tablet a6bc3c3d5d904830bfda3966e47305da) from {username='slave'} at 127.0.0.1:39626
I20260811 01:30:59.833885 7006 tablet_service.cc:1467] Processing CreateTablet for tablet 1cb26d431b24475a8cf62325267b0a3d (DEFAULT_TABLE table=default.table_c93618b4070e4974a6a794087d73fa73 [id=a65d01fd84fa41a3b31bf35b60f5cbaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:30:59.834540 7006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1cb26d431b24475a8cf62325267b0a3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:30:59.835191 7557 ts_tablet_manager.cc:1905] T b31aa7bcb4b249bd931573f48ca253de P 10a02c605db44ec69c67a34f53c2a055: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:30:59.838130 6004 catalog_manager.cc:4841] TS 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): tablet 8d33739d8c014d8686784505ff5f13ab (table default.table_f7f5b09418c242d092b1948b866994e5 [id=69f9a5a340894b4bb41b150af86e0d8a]) successfully deleted
I20260811 01:30:59.838703 6004 catalog_manager.cc:4841] TS 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): tablet b31aa7bcb4b249bd931573f48ca253de (table default.table_c93618b4070e4974a6a794087d73fa73 [id=a65d01fd84fa41a3b31bf35b60f5cbaf]) successfully deleted
I20260811 01:30:59.833259 7008 tablet_service.cc:1514] Processing DeleteTablet for tablet b31aa7bcb4b249bd931573f48ca253de with delete_type TABLET_DATA_DELETED (replaced by tablet 1cb26d431b24475a8cf62325267b0a3d) from {username='slave'} at 127.0.0.1:39626
I20260811 01:30:59.833320 7007 tablet_service.cc:1467] Processing CreateTablet for tablet a6bc3c3d5d904830bfda3966e47305da (DEFAULT_TABLE table=default.table_f7f5b09418c242d092b1948b866994e5 [id=69f9a5a340894b4bb41b150af86e0d8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:30:59.840461 7007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6bc3c3d5d904830bfda3966e47305da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:30:59.840590 7571 tablet_replica.cc:330] T b31aa7bcb4b249bd931573f48ca253de P 992432fc73c94fcc9f2142b27ef725af: stopping tablet replica
I20260811 01:30:59.840899 7571 raft_consensus.cc:2229] T b31aa7bcb4b249bd931573f48ca253de P 992432fc73c94fcc9f2142b27ef725af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:30:59.841289 7571 raft_consensus.cc:2258] T b31aa7bcb4b249bd931573f48ca253de P 992432fc73c94fcc9f2142b27ef725af [term 1 FOLLOWER]: Raft consensus is shut down!
@ 0x7fddd485566c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260811 01:30:59.848729 7567 tablet_bootstrap.cc:492] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055: Bootstrap starting.
I20260811 01:30:59.851349 7567 tablet_bootstrap.cc:654] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055: Neither blocks nor log segments found. Creating new log.
I20260811 01:30:59.853957 7571 ts_tablet_manager.cc:1905] T b31aa7bcb4b249bd931573f48ca253de P 992432fc73c94fcc9f2142b27ef725af: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7fddd48554e9 std::_Function_handler<>::_M_invoke() at ??:0
I20260811 01:30:59.855875 7567 tablet_bootstrap.cc:492] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055: No bootstrap required, opened a new log
I20260811 01:30:59.856055 7567 ts_tablet_manager.cc:1397] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
@ 0x9edb7d std::function<>::operator()() at ??:?
I20260811 01:30:59.857502 7564 tablet_bootstrap.cc:492] T a6bc3c3d5d904830bfda3966e47305da P 4a331de1488e4a0986b490917efea000: No bootstrap required, opened a new log
I20260811 01:30:59.857685 7564 ts_tablet_manager.cc:1397] T a6bc3c3d5d904830bfda3966e47305da P 4a331de1488e4a0986b490917efea000: Time spent bootstrapping tablet: real 0.050s user 0.007s sys 0.004s
I20260811 01:30:59.858708 7564 raft_consensus.cc:348] T a6bc3c3d5d904830bfda3966e47305da P 4a331de1488e4a0986b490917efea000 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:30:59.859030 7564 raft_consensus.cc:374] T a6bc3c3d5d904830bfda3966e47305da P 4a331de1488e4a0986b490917efea000 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:30:59.859128 7564 raft_consensus.cc:729] T a6bc3c3d5d904830bfda3966e47305da P 4a331de1488e4a0986b490917efea000 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a331de1488e4a0986b490917efea000, State: Initialized, Role: FOLLOWER
I20260811 01:30:59.859926 7567 raft_consensus.cc:348] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:30:59.860270 7567 raft_consensus.cc:374] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:30:59.860389 7567 raft_consensus.cc:729] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10a02c605db44ec69c67a34f53c2a055, State: Initialized, Role: FOLLOWER
I20260811 01:30:59.860749 7567 consensus_queue.cc:260] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
@ 0x7fddc18dbeaf kudu::Thread::SuperviseThread() at ??:0
I20260811 01:30:59.862099 7564 consensus_queue.cc:260] T a6bc3c3d5d904830bfda3966e47305da P 4a331de1488e4a0986b490917efea000 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:30:59.862411 7576 tablet_bootstrap.cc:492] T a6bc3c3d5d904830bfda3966e47305da P 992432fc73c94fcc9f2142b27ef725af: Bootstrap starting.
I20260811 01:30:59.864394 7576 tablet_bootstrap.cc:654] T a6bc3c3d5d904830bfda3966e47305da P 992432fc73c94fcc9f2142b27ef725af: Neither blocks nor log segments found. Creating new log.
I20260811 01:30:59.865240 7557 ts_tablet_manager.cc:1918] T b31aa7bcb4b249bd931573f48ca253de P 10a02c605db44ec69c67a34f53c2a055: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 01:30:59.865404 7557 log.cc:1196] T b31aa7bcb4b249bd931573f48ca253de P 10a02c605db44ec69c67a34f53c2a055: Deleting WAL directory at /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-1/wal/wals/b31aa7bcb4b249bd931573f48ca253de
I20260811 01:30:59.865963 7557 ts_tablet_manager.cc:1939] T b31aa7bcb4b249bd931573f48ca253de P 10a02c605db44ec69c67a34f53c2a055: Deleting consensus metadata
I20260811 01:30:59.867563 5999 catalog_manager.cc:4841] TS 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): tablet b31aa7bcb4b249bd931573f48ca253de (table default.table_c93618b4070e4974a6a794087d73fa73 [id=a65d01fd84fa41a3b31bf35b60f5cbaf]) successfully deleted
@ 0x7fddc4bb56db start_thread at ??:0
@ 0x7fddbdb4b61f clone at ??:0
I20260811 01:30:59.877763 7576 tablet_bootstrap.cc:492] T a6bc3c3d5d904830bfda3966e47305da P 992432fc73c94fcc9f2142b27ef725af: No bootstrap required, opened a new log
I20260811 01:30:59.877995 7576 ts_tablet_manager.cc:1397] T a6bc3c3d5d904830bfda3966e47305da P 992432fc73c94fcc9f2142b27ef725af: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.005s
I20260811 01:30:59.879097 7576 raft_consensus.cc:348] T a6bc3c3d5d904830bfda3966e47305da P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:30:59.879431 7576 raft_consensus.cc:374] T a6bc3c3d5d904830bfda3966e47305da P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:30:59.879541 7576 raft_consensus.cc:729] T a6bc3c3d5d904830bfda3966e47305da P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 992432fc73c94fcc9f2142b27ef725af, State: Initialized, Role: FOLLOWER
I20260811 01:30:59.879825 7576 consensus_queue.cc:260] T a6bc3c3d5d904830bfda3966e47305da P 992432fc73c94fcc9f2142b27ef725af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:30:59.883188 7590 raft_consensus.cc:482] T a6bc3c3d5d904830bfda3966e47305da P 4a331de1488e4a0986b490917efea000 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:30:59.883410 7590 raft_consensus.cc:504] T a6bc3c3d5d904830bfda3966e47305da P 4a331de1488e4a0986b490917efea000 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:30:59.884162 7571 ts_tablet_manager.cc:1918] T b31aa7bcb4b249bd931573f48ca253de P 992432fc73c94fcc9f2142b27ef725af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 01:30:59.884209 7590 leader_election.cc:288] T a6bc3c3d5d904830bfda3966e47305da P 4a331de1488e4a0986b490917efea000 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433), 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971)
I20260811 01:30:59.884367 7571 log.cc:1196] T b31aa7bcb4b249bd931573f48ca253de P 992432fc73c94fcc9f2142b27ef725af: Deleting WAL directory at /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-2/wal/wals/b31aa7bcb4b249bd931573f48ca253de
I20260811 01:30:59.884955 7571 ts_tablet_manager.cc:1939] T b31aa7bcb4b249bd931573f48ca253de P 992432fc73c94fcc9f2142b27ef725af: Deleting consensus metadata
I20260811 01:30:59.884878 7036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6bc3c3d5d904830bfda3966e47305da" candidate_uuid: "4a331de1488e4a0986b490917efea000" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "992432fc73c94fcc9f2142b27ef725af" is_pre_election: true
I20260811 01:30:59.885291 7036 raft_consensus.cc:2454] T a6bc3c3d5d904830bfda3966e47305da P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a331de1488e4a0986b490917efea000 in term 0.
I20260811 01:30:59.886221 7576 ts_tablet_manager.cc:1428] T a6bc3c3d5d904830bfda3966e47305da P 992432fc73c94fcc9f2142b27ef725af: Time spent starting tablet: real 0.008s user 0.003s sys 0.000s
I20260811 01:30:59.886250 6470 leader_election.cc:302] T a6bc3c3d5d904830bfda3966e47305da P 4a331de1488e4a0986b490917efea000 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af; no voters:
I20260811 01:30:59.887488 7571 tablet_replica.cc:330] T 8d33739d8c014d8686784505ff5f13ab P 992432fc73c94fcc9f2142b27ef725af: stopping tablet replica
I20260811 01:30:59.887537 6804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6bc3c3d5d904830bfda3966e47305da" candidate_uuid: "4a331de1488e4a0986b490917efea000" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10a02c605db44ec69c67a34f53c2a055" is_pre_election: true
I20260811 01:30:59.887734 7571 raft_consensus.cc:2229] T 8d33739d8c014d8686784505ff5f13ab P 992432fc73c94fcc9f2142b27ef725af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:30:59.887961 7571 raft_consensus.cc:2258] T 8d33739d8c014d8686784505ff5f13ab P 992432fc73c94fcc9f2142b27ef725af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:30:59.888216 7590 raft_consensus.cc:2781] T a6bc3c3d5d904830bfda3966e47305da P 4a331de1488e4a0986b490917efea000 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:30:59.888351 7590 raft_consensus.cc:482] T a6bc3c3d5d904830bfda3966e47305da P 4a331de1488e4a0986b490917efea000 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:30:59.888447 7590 raft_consensus.cc:3037] T a6bc3c3d5d904830bfda3966e47305da P 4a331de1488e4a0986b490917efea000 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:30:59.889479 7571 ts_tablet_manager.cc:1905] T 8d33739d8c014d8686784505ff5f13ab P 992432fc73c94fcc9f2142b27ef725af: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 01:30:59.890765 6469 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 4a331de1488e4a0986b490917efea000 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Illegal state: must be running to vote when last-logged opid is not known
I20260811 01:30:59.891386 7567 ts_tablet_manager.cc:1428] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055: Time spent starting tablet: real 0.035s user 0.000s sys 0.008s
I20260811 01:30:59.891306 7590 raft_consensus.cc:504] T a6bc3c3d5d904830bfda3966e47305da P 4a331de1488e4a0986b490917efea000 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:30:59.892310 7576 tablet_bootstrap.cc:492] T 1cb26d431b24475a8cf62325267b0a3d P 992432fc73c94fcc9f2142b27ef725af: Bootstrap starting.
I20260811 01:30:59.892736 7036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6bc3c3d5d904830bfda3966e47305da" candidate_uuid: "4a331de1488e4a0986b490917efea000" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "992432fc73c94fcc9f2142b27ef725af"
I20260811 01:30:59.892930 7036 raft_consensus.cc:3037] T a6bc3c3d5d904830bfda3966e47305da P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:30:59.894387 7576 tablet_bootstrap.cc:654] T 1cb26d431b24475a8cf62325267b0a3d P 992432fc73c94fcc9f2142b27ef725af: Neither blocks nor log segments found. Creating new log.
I20260811 01:30:59.895594 7590 leader_election.cc:288] T a6bc3c3d5d904830bfda3966e47305da P 4a331de1488e4a0986b490917efea000 [CANDIDATE]: Term 1 election: Requested vote from peers 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433), 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971)
I20260811 01:30:59.896354 7036 raft_consensus.cc:2454] T a6bc3c3d5d904830bfda3966e47305da P 992432fc73c94fcc9f2142b27ef725af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a331de1488e4a0986b490917efea000 in term 1.
I20260811 01:30:59.897423 6470 leader_election.cc:302] T a6bc3c3d5d904830bfda3966e47305da P 4a331de1488e4a0986b490917efea000 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af; no voters:
I20260811 01:30:59.897733 7571 ts_tablet_manager.cc:1918] T 8d33739d8c014d8686784505ff5f13ab P 992432fc73c94fcc9f2142b27ef725af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 01:30:59.897938 7571 log.cc:1196] T 8d33739d8c014d8686784505ff5f13ab P 992432fc73c94fcc9f2142b27ef725af: Deleting WAL directory at /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-2/wal/wals/8d33739d8c014d8686784505ff5f13ab
I20260811 01:30:59.898541 7571 ts_tablet_manager.cc:1939] T 8d33739d8c014d8686784505ff5f13ab P 992432fc73c94fcc9f2142b27ef725af: Deleting consensus metadata
I20260811 01:30:59.900328 7590 raft_consensus.cc:2781] T a6bc3c3d5d904830bfda3966e47305da P 4a331de1488e4a0986b490917efea000 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:30:59.900504 7590 raft_consensus.cc:686] T a6bc3c3d5d904830bfda3966e47305da P 4a331de1488e4a0986b490917efea000 [term 1 LEADER]: Becoming Leader. State: Replica: 4a331de1488e4a0986b490917efea000, State: Running, Role: LEADER
I20260811 01:30:59.900870 7590 consensus_queue.cc:237] T a6bc3c3d5d904830bfda3966e47305da P 4a331de1488e4a0986b490917efea000 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:30:59.903383 6804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6bc3c3d5d904830bfda3966e47305da" candidate_uuid: "4a331de1488e4a0986b490917efea000" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10a02c605db44ec69c67a34f53c2a055"
W20260811 01:30:59.904227 6469 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 4a331de1488e4a0986b490917efea000 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Illegal state: must be running to vote when last-logged opid is not known
I20260811 01:30:59.908866 7567 tablet_bootstrap.cc:492] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055: Bootstrap starting.
I20260811 01:30:59.911933 7564 ts_tablet_manager.cc:1428] T a6bc3c3d5d904830bfda3966e47305da P 4a331de1488e4a0986b490917efea000: Time spent starting tablet: real 0.054s user 0.008s sys 0.000s
I20260811 01:30:59.914204 7564 tablet_bootstrap.cc:492] T 1cb26d431b24475a8cf62325267b0a3d P 4a331de1488e4a0986b490917efea000: Bootstrap starting.
I20260811 01:30:59.916518 7564 tablet_bootstrap.cc:654] T 1cb26d431b24475a8cf62325267b0a3d P 4a331de1488e4a0986b490917efea000: Neither blocks nor log segments found. Creating new log.
I20260811 01:30:59.918184 7567 tablet_bootstrap.cc:654] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055: Neither blocks nor log segments found. Creating new log.
I20260811 01:30:59.920352 7564 tablet_bootstrap.cc:492] T 1cb26d431b24475a8cf62325267b0a3d P 4a331de1488e4a0986b490917efea000: No bootstrap required, opened a new log
I20260811 01:30:59.920550 7564 ts_tablet_manager.cc:1397] T 1cb26d431b24475a8cf62325267b0a3d P 4a331de1488e4a0986b490917efea000: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.001s
I20260811 01:30:59.921461 7564 raft_consensus.cc:348] T 1cb26d431b24475a8cf62325267b0a3d P 4a331de1488e4a0986b490917efea000 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:30:59.921741 7564 raft_consensus.cc:374] T 1cb26d431b24475a8cf62325267b0a3d P 4a331de1488e4a0986b490917efea000 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:30:59.921824 7564 raft_consensus.cc:729] T 1cb26d431b24475a8cf62325267b0a3d P 4a331de1488e4a0986b490917efea000 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a331de1488e4a0986b490917efea000, State: Initialized, Role: FOLLOWER
I20260811 01:30:59.922168 7564 consensus_queue.cc:260] T 1cb26d431b24475a8cf62325267b0a3d P 4a331de1488e4a0986b490917efea000 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:30:59.922919 7564 ts_tablet_manager.cc:1428] T 1cb26d431b24475a8cf62325267b0a3d P 4a331de1488e4a0986b490917efea000: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 01:30:59.924758 7576 tablet_bootstrap.cc:492] T 1cb26d431b24475a8cf62325267b0a3d P 992432fc73c94fcc9f2142b27ef725af: No bootstrap required, opened a new log
I20260811 01:30:59.924943 7576 ts_tablet_manager.cc:1397] T 1cb26d431b24475a8cf62325267b0a3d P 992432fc73c94fcc9f2142b27ef725af: Time spent bootstrapping tablet: real 0.033s user 0.013s sys 0.017s
I20260811 01:30:59.925804 7576 raft_consensus.cc:348] T 1cb26d431b24475a8cf62325267b0a3d P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:30:59.926095 7576 raft_consensus.cc:374] T 1cb26d431b24475a8cf62325267b0a3d P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:30:59.926182 7576 raft_consensus.cc:729] T 1cb26d431b24475a8cf62325267b0a3d P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 992432fc73c94fcc9f2142b27ef725af, State: Initialized, Role: FOLLOWER
I20260811 01:30:59.926465 7576 consensus_queue.cc:260] T 1cb26d431b24475a8cf62325267b0a3d P 992432fc73c94fcc9f2142b27ef725af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:30:59.927069 7576 ts_tablet_manager.cc:1428] T 1cb26d431b24475a8cf62325267b0a3d P 992432fc73c94fcc9f2142b27ef725af: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 01:30:59.927829 7567 tablet_bootstrap.cc:492] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055: No bootstrap required, opened a new log
I20260811 01:30:59.928037 7567 ts_tablet_manager.cc:1397] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.000s
I20260811 01:30:59.928917 7567 raft_consensus.cc:348] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:30:59.929208 7567 raft_consensus.cc:374] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:30:59.929293 7567 raft_consensus.cc:729] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10a02c605db44ec69c67a34f53c2a055, State: Initialized, Role: FOLLOWER
I20260811 01:30:59.930250 7567 consensus_queue.cc:260] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:30:59.930900 7567 ts_tablet_manager.cc:1428] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
W20260811 01:31:00.107124 6927 connection.cc:535] server connection from 127.0.0.1:39626 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 01:31:00.107422 7129 connection.cc:535] client connection to 127.5.107.254:33755 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 01:31:00.108173 7129 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.5.107.254:33755)
W20260811 01:31:00.108923 7129 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.5.107.254:33755)
W20260811 01:31:00.109162 7129 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.5.107.254:33755)
W20260811 01:31:00.109812 7552 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:31:00.108203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.107.254:33755 (local address 127.5.107.195:37671)
0811 01:31:00.108604 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0811 01:31:00.108718 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.107.254:33755: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":179}
I20260811 01:31:00.182860 7585 raft_consensus.cc:482] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:00.183202 7585 raft_consensus.cc:504] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:31:00.185191 7585 leader_election.cc:288] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
I20260811 01:31:00.193363 7036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6bc3c3d5d904830bfda3966e47305da" candidate_uuid: "10a02c605db44ec69c67a34f53c2a055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "992432fc73c94fcc9f2142b27ef725af" is_pre_election: true
I20260811 01:31:00.193513 6590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6bc3c3d5d904830bfda3966e47305da" candidate_uuid: "10a02c605db44ec69c67a34f53c2a055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a331de1488e4a0986b490917efea000" is_pre_election: true
I20260811 01:31:00.193729 7036 raft_consensus.cc:2379] T a6bc3c3d5d904830bfda3966e47305da P 992432fc73c94fcc9f2142b27ef725af [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10a02c605db44ec69c67a34f53c2a055 in current term 1: Already voted for candidate 4a331de1488e4a0986b490917efea000 in this term.
I20260811 01:31:00.194602 6702 leader_election.cc:302] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:31:00.195006 7585 raft_consensus.cc:3037] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:31:00.197119 7585 raft_consensus.cc:2726] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:31:00.324173 7618 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 61c27cf3d4c846748ae6752e0fa652b9)
I20260811 01:31:00.324477 7618 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:00.326434 7618 leader_election.cc:288] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755), 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427)
W20260811 01:31:00.327436 7323 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.107.254:33755: connect: Connection refused (error 111)
W20260811 01:31:00.328812 7326 leader_election.cc:334] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:31:00.329079 7323 leader_election.cc:334] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755): Network error: Client connection negotiation failed: client connection to 127.5.107.254:33755: connect: Connection refused (error 111)
I20260811 01:31:00.329234 7323 leader_election.cc:302] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c6f417c29b464762944a721d477b2b13; no voters: 61c27cf3d4c846748ae6752e0fa652b9, 8970a89e5fd744008240ecd668272a4a
I20260811 01:31:00.329638 7618 raft_consensus.cc:2726] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260811 01:31:00.341472 7592 raft_consensus.cc:482] T 1cb26d431b24475a8cf62325267b0a3d P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:00.341801 7592 raft_consensus.cc:504] T 1cb26d431b24475a8cf62325267b0a3d P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:31:00.343743 7592 leader_election.cc:288] T 1cb26d431b24475a8cf62325267b0a3d P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971)
I20260811 01:31:00.350018 6590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cb26d431b24475a8cf62325267b0a3d" candidate_uuid: "992432fc73c94fcc9f2142b27ef725af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a331de1488e4a0986b490917efea000" is_pre_election: true
I20260811 01:31:00.350435 6590 raft_consensus.cc:2454] T 1cb26d431b24475a8cf62325267b0a3d P 4a331de1488e4a0986b490917efea000 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992432fc73c94fcc9f2142b27ef725af in term 0.
I20260811 01:31:00.351006 6923 leader_election.cc:302] T 1cb26d431b24475a8cf62325267b0a3d P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af; no voters:
I20260811 01:31:00.351104 6804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cb26d431b24475a8cf62325267b0a3d" candidate_uuid: "992432fc73c94fcc9f2142b27ef725af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10a02c605db44ec69c67a34f53c2a055" is_pre_election: true
I20260811 01:31:00.351366 6804 raft_consensus.cc:2454] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 992432fc73c94fcc9f2142b27ef725af in term 0.
I20260811 01:31:00.351428 7592 raft_consensus.cc:2781] T 1cb26d431b24475a8cf62325267b0a3d P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:31:00.351573 7592 raft_consensus.cc:482] T 1cb26d431b24475a8cf62325267b0a3d P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:31:00.351711 7592 raft_consensus.cc:3037] T 1cb26d431b24475a8cf62325267b0a3d P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:31:00.354122 7592 raft_consensus.cc:504] T 1cb26d431b24475a8cf62325267b0a3d P 992432fc73c94fcc9f2142b27ef725af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:31:00.354913 7592 leader_election.cc:288] T 1cb26d431b24475a8cf62325267b0a3d P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 election: Requested vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971)
I20260811 01:31:00.355645 6590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cb26d431b24475a8cf62325267b0a3d" candidate_uuid: "992432fc73c94fcc9f2142b27ef725af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a331de1488e4a0986b490917efea000"
I20260811 01:31:00.355729 6804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cb26d431b24475a8cf62325267b0a3d" candidate_uuid: "992432fc73c94fcc9f2142b27ef725af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10a02c605db44ec69c67a34f53c2a055"
I20260811 01:31:00.355914 6590 raft_consensus.cc:3037] T 1cb26d431b24475a8cf62325267b0a3d P 4a331de1488e4a0986b490917efea000 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:31:00.355960 6804 raft_consensus.cc:3037] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:31:00.358260 6590 raft_consensus.cc:2454] T 1cb26d431b24475a8cf62325267b0a3d P 4a331de1488e4a0986b490917efea000 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992432fc73c94fcc9f2142b27ef725af in term 1.
I20260811 01:31:00.358742 6804 raft_consensus.cc:2454] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 992432fc73c94fcc9f2142b27ef725af in term 1.
I20260811 01:31:00.358781 6923 leader_election.cc:302] T 1cb26d431b24475a8cf62325267b0a3d P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af; no voters:
I20260811 01:31:00.359184 7592 raft_consensus.cc:2781] T 1cb26d431b24475a8cf62325267b0a3d P 992432fc73c94fcc9f2142b27ef725af [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:31:00.359784 7592 raft_consensus.cc:686] T 1cb26d431b24475a8cf62325267b0a3d P 992432fc73c94fcc9f2142b27ef725af [term 1 LEADER]: Becoming Leader. State: Replica: 992432fc73c94fcc9f2142b27ef725af, State: Running, Role: LEADER
I20260811 01:31:00.360219 7592 consensus_queue.cc:237] T 1cb26d431b24475a8cf62325267b0a3d P 992432fc73c94fcc9f2142b27ef725af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:31:00.724789 5551 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZLXb2N/build/asan/bin/kudu pid 7216
I20260811 01:31:00.739974 7418 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:31:00.757308 7634 consensus_queue.cc:1035] T a6bc3c3d5d904830bfda3966e47305da P 4a331de1488e4a0986b490917efea000 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:31:00.767138 7645 consensus_queue.cc:1035] T a6bc3c3d5d904830bfda3966e47305da P 4a331de1488e4a0986b490917efea000 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:31:00.871812 5551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZLXb2N/build/asan/bin/kudu with pid 7216
W20260811 01:31:00.906658 7624 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:31:00.905683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.107.252:45519 (local address 127.5.107.195:43971)
0811 01:31:00.905968 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 01:31:00.905994 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 01:31:00.906185 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:31:00.906412 (+ 227us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":94}
W20260811 01:31:00.907375 7648 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:31:00.906178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.107.252:45519 (local address 127.5.107.194:48943)
0811 01:31:00.906423 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 01:31:00.906536 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 01:31:01.277581 7592 consensus_queue.cc:1035] T 1cb26d431b24475a8cf62325267b0a3d P 992432fc73c94fcc9f2142b27ef725af [LEADER]: Connected to new peer: Peer: permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:31:01.287473 7666 consensus_queue.cc:1035] T 1cb26d431b24475a8cf62325267b0a3d P 992432fc73c94fcc9f2142b27ef725af [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260811 01:31:01.616205 5551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZLXb2N/build/asan/bin/kudu
/tmp/dist-test-taskZLXb2N/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.107.252:45519
--webserver_interface=127.5.107.252
--webserver_port=39897
--builtin_ntp_servers=127.5.107.212:36421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519
--hive_metastore_uris=thrift://127.0.0.1:36237
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260811 01:31:01.973284 7677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:31:01.973676 7677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:31:01.973819 7677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:31:01.982424 7677 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:31:01.982576 7677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:31:01.982672 7677 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:31:01.982775 7677 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:31:01.992700 7677 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.107.212:36421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:36237
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.107.252:45519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.107.252
--webserver_port=39897
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:31:01.994240 7677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:31:01.996073 7677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:31:02.008611 7693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:31:02.008611 7699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:31:02.008637 7696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:31:02.014905 7677 server_base.cc:1034] running on GCE node
I20260811 01:31:02.016132 7677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:31:02.017932 7677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:31:02.019200 7677 hybrid_clock.cc:648] HybridClock initialized: now 1786411862019114 us; error 56 us; skew 500 ppm
I20260811 01:31:02.019563 7677 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:31:02.022042 7677 webserver.cc:460] Webserver started at http://127.5.107.252:39897/ using document root <none> and password file <none>
I20260811 01:31:02.022847 7677 fs_manager.cc:359] Metadata directory not provided
I20260811 01:31:02.023002 7677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:31:02.028700 7677 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260811 01:31:02.031802 7716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:31:02.033210 7677 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 01:31:02.033406 7677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/data,/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/wal
uuid: "c6f417c29b464762944a721d477b2b13"
format_stamp: "Formatted at 2026-08-11 01:30:54 on dist-test-slave-2p8l"
I20260811 01:31:02.034451 7677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:31:02.052189 7677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:31:02.053145 7677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:31:02.053467 7677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:31:02.071923 7677 rpc_server.cc:307] RPC server started. Bound to: 127.5.107.252:45519
I20260811 01:31:02.071945 7808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.107.252:45519 every 8 connection(s)
I20260811 01:31:02.073560 7677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/data/info.pb
I20260811 01:31:02.080495 5551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZLXb2N/build/asan/bin/kudu as pid 7677
I20260811 01:31:02.081831 7809 sys_catalog.cc:263] Verifying existing consensus state
I20260811 01:31:02.087608 7809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13: Bootstrap starting.
I20260811 01:31:02.100112 7809 log.cc:824] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13: Log is configured to *not* fsync() on all Append() calls
I20260811 01:31:02.120450 7809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13: Bootstrap replayed 1/1 log segments. Stats: ops{read=19 overwritten=0 applied=19 ignored=0} inserts{seen=12 ignored=0} mutations{seen=11 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 01:31:02.121006 7809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13: Bootstrap complete.
I20260811 01:31:02.128860 7809 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:02.129814 7809 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6f417c29b464762944a721d477b2b13, State: Initialized, Role: FOLLOWER
I20260811 01:31:02.130676 7809 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 19, Last appended: 3.19, Last appended by leader: 19, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:02.131623 7815 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } } }
I20260811 01:31:02.132002 7815 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:31:02.132848 7809 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:31:02.148869 7841 catalog_manager.cc:1259] Loaded cluster ID: 2f0039b0f9ec4f479f59cf7d8b74b7a2
I20260811 01:31:02.149120 7841 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13: loading cluster ID for follower catalog manager: success
I20260811 01:31:02.143337 7677 master_runner.cc:186] Error getting master registration for 127.5.107.254:33755: Network error: Client connection negotiation failed: client connection to 127.5.107.254:33755: connect: Connection refused (error 111),
I20260811 01:31:02.153674 7677 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:31:02.153868 7841 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13: acquiring CA information for follower catalog manager: success
I20260811 01:31:02.156780 7841 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 01:31:02.314301 6890 heartbeater.cc:347] Connected to a master server at 127.5.107.252:45519
I20260811 01:31:02.316383 7110 heartbeater.cc:347] Connected to a master server at 127.5.107.252:45519
I20260811 01:31:02.318511 7749 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" instance_seqno: 1786411855798958) as {username='slave'} at 127.5.107.194:41681; Asking this server to re-register.
I20260811 01:31:02.318488 7748 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" instance_seqno: 1786411856431448) as {username='slave'} at 127.5.107.195:36619; Asking this server to re-register.
I20260811 01:31:02.319418 7110 heartbeater.cc:464] Registering TS with master...
I20260811 01:31:02.319427 6890 heartbeater.cc:464] Registering TS with master...
I20260811 01:31:02.321147 7748 ts_manager.cc:194] Registered new tserver with Master: 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
I20260811 01:31:02.321271 7749 ts_manager.cc:194] Registered new tserver with Master: 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971)
I20260811 01:31:02.513185 7815 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:02.513571 7815 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
W20260811 01:31:02.516968 7718 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.107.254:33755: connect: Connection refused (error 111)
I20260811 01:31:02.517786 7815 leader_election.cc:288] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755), 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427)
W20260811 01:31:02.519939 7719 leader_election.cc:334] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:31:02.521309 7718 leader_election.cc:334] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755): Network error: Client connection negotiation failed: client connection to 127.5.107.254:33755: connect: Connection refused (error 111)
I20260811 01:31:02.521517 7718 leader_election.cc:302] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c6f417c29b464762944a721d477b2b13; no voters: 61c27cf3d4c846748ae6752e0fa652b9, 8970a89e5fd744008240ecd668272a4a
I20260811 01:31:02.521991 7815 raft_consensus.cc:2726] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260811 01:31:02.527245 6654 heartbeater.cc:347] Connected to a master server at 127.5.107.252:45519
I20260811 01:31:02.528988 7749 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "4a331de1488e4a0986b490917efea000" instance_seqno: 1786411855240795) as {username='slave'} at 127.5.107.193:38633; Asking this server to re-register.
I20260811 01:31:02.529615 6654 heartbeater.cc:464] Registering TS with master...
I20260811 01:31:02.530925 7749 ts_manager.cc:194] Registered new tserver with Master: 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
I20260811 01:31:03.207971 7867 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:03.208243 7867 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:03.209105 7867 leader_election.cc:288] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755), 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427)
W20260811 01:31:03.211458 7719 leader_election.cc:334] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:31:03.211628 7718 leader_election.cc:334] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755): Network error: Client connection negotiation failed: client connection to 127.5.107.254:33755: connect: Connection refused (error 111)
I20260811 01:31:03.211758 7718 leader_election.cc:302] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c6f417c29b464762944a721d477b2b13; no voters: 61c27cf3d4c846748ae6752e0fa652b9, 8970a89e5fd744008240ecd668272a4a
I20260811 01:31:03.212019 7867 raft_consensus.cc:2726] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260811 01:31:04.223349 5551 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZLXb2N/build/asan/bin/kudu pid 7677
I20260811 01:31:04.231710 7804 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:31:04.341023 7890 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:04.341605 5551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZLXb2N/build/asan/bin/kudu with pid 7677
I20260811 01:31:04.341449 7890 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:05.029217 5551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZLXb2N/build/asan/bin/kudu
/tmp/dist-test-taskZLXb2N/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.107.252:45519
--webserver_interface=127.5.107.252
--webserver_port=39897
--builtin_ntp_servers=127.5.107.212:36421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519
--hive_metastore_uris=thrift://127.0.0.1:36237
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260811 01:31:05.392413 7901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:31:05.392755 7901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:31:05.392932 7901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:31:05.401073 7901 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:31:05.401191 7901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:31:05.401289 7901 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:31:05.401383 7901 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:31:05.411348 7901 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.107.212:36421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:36237
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.107.252:45519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.107.252
--webserver_port=39897
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:31:05.412835 7901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:31:05.414799 7901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:31:05.427279 7910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:31:05.427327 7914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:31:05.427237 7912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:31:05.428284 7901 server_base.cc:1034] running on GCE node
I20260811 01:31:05.429237 7901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:31:05.430868 7901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:31:05.432147 7901 hybrid_clock.cc:648] HybridClock initialized: now 1786411865432059 us; error 72 us; skew 500 ppm
I20260811 01:31:05.432548 7901 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:31:05.435343 7901 webserver.cc:460] Webserver started at http://127.5.107.252:39897/ using document root <none> and password file <none>
I20260811 01:31:05.436162 7901 fs_manager.cc:359] Metadata directory not provided
I20260811 01:31:05.436339 7901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:31:05.442948 7901 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 01:31:05.446944 7924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:31:05.448423 7901 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 01:31:05.448697 7901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/data,/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/wal
uuid: "c6f417c29b464762944a721d477b2b13"
format_stamp: "Formatted at 2026-08-11 01:30:54 on dist-test-slave-2p8l"
I20260811 01:31:05.449762 7901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:31:05.478168 7901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:31:05.479161 7901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:31:05.479485 7901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:31:05.501183 7901 rpc_server.cc:307] RPC server started. Bound to: 127.5.107.252:45519
I20260811 01:31:05.501215 8043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.107.252:45519 every 8 connection(s)
I20260811 01:31:05.503535 7901 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/data/info.pb
I20260811 01:31:05.504910 5551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZLXb2N/build/asan/bin/kudu as pid 7901
I20260811 01:31:05.513202 8046 sys_catalog.cc:263] Verifying existing consensus state
I20260811 01:31:05.518783 8046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13: Bootstrap starting.
I20260811 01:31:05.536075 6654 heartbeater.cc:347] Connected to a master server at 127.5.107.252:45519
I20260811 01:31:05.536556 8046 log.cc:824] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13: Log is configured to *not* fsync() on all Append() calls
I20260811 01:31:05.557548 8046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13: Bootstrap replayed 1/1 log segments. Stats: ops{read=19 overwritten=0 applied=19 ignored=0} inserts{seen=12 ignored=0} mutations{seen=11 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 01:31:05.558310 8046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13: Bootstrap complete.
I20260811 01:31:05.566792 8046 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:05.567863 8046 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6f417c29b464762944a721d477b2b13, State: Initialized, Role: FOLLOWER
I20260811 01:31:05.568612 8046 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 19, Last appended: 3.19, Last appended by leader: 19, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:05.569538 8066 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } } }
I20260811 01:31:05.569869 8066 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:31:05.570873 8046 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:31:05.592633 8089 catalog_manager.cc:1259] Loaded cluster ID: 2f0039b0f9ec4f479f59cf7d8b74b7a2
I20260811 01:31:05.592816 8089 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13: loading cluster ID for follower catalog manager: success
I20260811 01:31:05.597247 8089 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13: acquiring CA information for follower catalog manager: success
I20260811 01:31:05.588423 7901 master_runner.cc:186] Error getting master registration for 127.5.107.254:33755: Network error: Client connection negotiation failed: client connection to 127.5.107.254:33755: connect: Connection refused (error 111),
I20260811 01:31:05.600337 8089 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 01:31:05.600526 7901 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:31:05.888757 6890 heartbeater.cc:347] Connected to a master server at 127.5.107.252:45519
I20260811 01:31:05.891307 7973 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" instance_seqno: 1786411855798958) as {username='slave'} at 127.5.107.194:37087; Asking this server to re-register.
I20260811 01:31:05.892067 6890 heartbeater.cc:464] Registering TS with master...
I20260811 01:31:05.893909 7973 ts_manager.cc:194] Registered new tserver with Master: 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971)
I20260811 01:31:05.948594 7110 heartbeater.cc:347] Connected to a master server at 127.5.107.252:45519
I20260811 01:31:05.950032 7973 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" instance_seqno: 1786411856431448) as {username='slave'} at 127.5.107.195:57253; Asking this server to re-register.
I20260811 01:31:05.950742 7110 heartbeater.cc:464] Registering TS with master...
I20260811 01:31:05.952183 7973 ts_manager.cc:194] Registered new tserver with Master: 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
I20260811 01:31:05.958688 8066 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:05.958989 8066 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:05.960896 8066 leader_election.cc:288] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755), 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427)
W20260811 01:31:05.961707 7933 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.107.254:33755: connect: Connection refused (error 111)
W20260811 01:31:05.963279 7937 leader_election.cc:334] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:31:05.963536 7933 leader_election.cc:334] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755): Network error: Client connection negotiation failed: client connection to 127.5.107.254:33755: connect: Connection refused (error 111)
I20260811 01:31:05.963735 7937 leader_election.cc:302] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c6f417c29b464762944a721d477b2b13; no voters: 61c27cf3d4c846748ae6752e0fa652b9, 8970a89e5fd744008240ecd668272a4a
I20260811 01:31:05.964157 8066 raft_consensus.cc:2726] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260811 01:31:06.040920 7973 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "4a331de1488e4a0986b490917efea000" instance_seqno: 1786411855240795) as {username='slave'} at 127.5.107.193:38719; Asking this server to re-register.
I20260811 01:31:06.041735 6654 heartbeater.cc:464] Registering TS with master...
I20260811 01:31:06.043103 7973 ts_manager.cc:194] Registered new tserver with Master: 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
I20260811 01:31:06.519464 8110 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:06.519665 8110 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:06.520399 8110 leader_election.cc:288] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755), 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427)
W20260811 01:31:06.522861 7933 leader_election.cc:334] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755): Network error: Client connection negotiation failed: client connection to 127.5.107.254:33755: connect: Connection refused (error 111)
W20260811 01:31:06.523088 7937 leader_election.cc:334] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
I20260811 01:31:06.523213 7937 leader_election.cc:302] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c6f417c29b464762944a721d477b2b13; no voters: 61c27cf3d4c846748ae6752e0fa652b9, 8970a89e5fd744008240ecd668272a4a
I20260811 01:31:06.523458 8110 raft_consensus.cc:2726] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260811 01:31:07.222800 8127 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:07.223013 8127 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:07.223685 8127 leader_election.cc:288] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755), 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427)
W20260811 01:31:07.225935 7933 leader_election.cc:334] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755): Network error: Client connection negotiation failed: client connection to 127.5.107.254:33755: connect: Connection refused (error 111)
W20260811 01:31:07.226168 7937 leader_election.cc:334] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
I20260811 01:31:07.226284 7937 leader_election.cc:302] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c6f417c29b464762944a721d477b2b13; no voters: 61c27cf3d4c846748ae6752e0fa652b9, 8970a89e5fd744008240ecd668272a4a
I20260811 01:31:07.226497 8127 raft_consensus.cc:2726] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260811 01:31:08.064883 5551 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZLXb2N/build/asan/bin/kudu pid 7901
I20260811 01:31:08.078716 8038 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:31:08.188220 5551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZLXb2N/build/asan/bin/kudu with pid 7901
I20260811 01:31:09.109707 5551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZLXb2N/build/asan/bin/kudu
/tmp/dist-test-taskZLXb2N/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.107.252:45519
--webserver_interface=127.5.107.252
--webserver_port=39897
--builtin_ntp_servers=127.5.107.212:36421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519
--hive_metastore_uris=thrift://127.0.0.1:36237
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260811 01:31:09.460976 8191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:31:09.461304 8191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:31:09.461472 8191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:31:09.469188 8191 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:31:09.469312 8191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:31:09.469415 8191 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:31:09.469518 8191 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:31:09.480589 8191 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.107.212:36421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:36237
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.107.252:45519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.107.252
--webserver_port=39897
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:31:09.481976 8191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:31:09.483907 8191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:31:09.496313 8205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:31:09.496877 8204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:31:09.497381 8209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:31:09.498732 8191 server_base.cc:1034] running on GCE node
I20260811 01:31:09.499522 8191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:31:09.501056 8191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:31:09.502291 8191 hybrid_clock.cc:648] HybridClock initialized: now 1786411869502204 us; error 58 us; skew 500 ppm
I20260811 01:31:09.502679 8191 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:31:09.505275 8191 webserver.cc:460] Webserver started at http://127.5.107.252:39897/ using document root <none> and password file <none>
I20260811 01:31:09.505967 8191 fs_manager.cc:359] Metadata directory not provided
I20260811 01:31:09.506136 8191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:31:09.511480 8191 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260811 01:31:09.514462 8222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:31:09.515834 8191 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 01:31:09.516039 8191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/data,/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/wal
uuid: "c6f417c29b464762944a721d477b2b13"
format_stamp: "Formatted at 2026-08-11 01:30:54 on dist-test-slave-2p8l"
I20260811 01:31:09.517042 8191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:31:09.530645 8191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:31:09.531626 8191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:31:09.531939 8191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:31:09.553536 8191 rpc_server.cc:307] RPC server started. Bound to: 127.5.107.252:45519
I20260811 01:31:09.553565 8312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.107.252:45519 every 8 connection(s)
I20260811 01:31:09.555223 8191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-2/data/info.pb
I20260811 01:31:09.562685 8313 sys_catalog.cc:263] Verifying existing consensus state
I20260811 01:31:09.564718 5551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZLXb2N/build/asan/bin/kudu as pid 8191
I20260811 01:31:09.568276 8313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13: Bootstrap starting.
I20260811 01:31:09.572502 6654 heartbeater.cc:347] Connected to a master server at 127.5.107.252:45519
I20260811 01:31:09.579957 8313 log.cc:824] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13: Log is configured to *not* fsync() on all Append() calls
I20260811 01:31:09.598196 8313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13: Bootstrap replayed 1/1 log segments. Stats: ops{read=19 overwritten=0 applied=19 ignored=0} inserts{seen=12 ignored=0} mutations{seen=11 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 01:31:09.598809 8313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13: Bootstrap complete.
I20260811 01:31:09.606678 8313 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:09.607642 8313 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6f417c29b464762944a721d477b2b13, State: Initialized, Role: FOLLOWER
I20260811 01:31:09.608274 8313 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 19, Last appended: 3.19, Last appended by leader: 19, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:09.609186 8320 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } } }
I20260811 01:31:09.609505 8320 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:31:09.610406 8313 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:31:09.625072 8344 catalog_manager.cc:1259] Loaded cluster ID: 2f0039b0f9ec4f479f59cf7d8b74b7a2
I20260811 01:31:09.625205 8344 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13: loading cluster ID for follower catalog manager: success
I20260811 01:31:09.620534 8191 master_runner.cc:186] Error getting master registration for 127.5.107.254:33755: Network error: Client connection negotiation failed: client connection to 127.5.107.254:33755: connect: Connection refused (error 111),
I20260811 01:31:09.630219 8191 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:31:09.632732 8344 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13: acquiring CA information for follower catalog manager: success
I20260811 01:31:09.635344 8344 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 01:31:09.917742 6890 heartbeater.cc:347] Connected to a master server at 127.5.107.252:45519
I20260811 01:31:09.920858 8253 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" instance_seqno: 1786411855798958) as {username='slave'} at 127.5.107.194:52921; Asking this server to re-register.
I20260811 01:31:09.921905 6890 heartbeater.cc:464] Registering TS with master...
I20260811 01:31:09.924062 8253 ts_manager.cc:194] Registered new tserver with Master: 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971)
I20260811 01:31:09.981248 7110 heartbeater.cc:347] Connected to a master server at 127.5.107.252:45519
I20260811 01:31:09.982985 8253 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" instance_seqno: 1786411856431448) as {username='slave'} at 127.5.107.195:48005; Asking this server to re-register.
I20260811 01:31:09.983727 7110 heartbeater.cc:464] Registering TS with master...
I20260811 01:31:09.985858 8253 ts_manager.cc:194] Registered new tserver with Master: 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
I20260811 01:31:09.998701 8320 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:09.999141 8320 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:10.002990 8320 leader_election.cc:288] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755), 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427)
W20260811 01:31:10.005393 8231 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:31:10.016745 8231 leader_election.cc:334] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:31:10.017208 8229 leader_election.cc:334] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755): Network error: Client connection negotiation failed: client connection to 127.5.107.254:33755: connect: Connection refused (error 111)
I20260811 01:31:10.017412 8229 leader_election.cc:302] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c6f417c29b464762944a721d477b2b13; no voters: 61c27cf3d4c846748ae6752e0fa652b9, 8970a89e5fd744008240ecd668272a4a
I20260811 01:31:10.017853 8320 raft_consensus.cc:2726] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260811 01:31:10.077142 8253 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "4a331de1488e4a0986b490917efea000" instance_seqno: 1786411855240795) as {username='slave'} at 127.5.107.193:47465; Asking this server to re-register.
I20260811 01:31:10.077903 6654 heartbeater.cc:464] Registering TS with master...
I20260811 01:31:10.079334 8253 ts_manager.cc:194] Registered new tserver with Master: 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
I20260811 01:31:10.580107 8372 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:10.580344 8372 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:10.581110 8372 leader_election.cc:288] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755), 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427)
W20260811 01:31:10.583578 8231 leader_election.cc:334] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:31:10.583773 8229 leader_election.cc:334] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755): Network error: Client connection negotiation failed: client connection to 127.5.107.254:33755: connect: Connection refused (error 111)
I20260811 01:31:10.583904 8229 leader_election.cc:302] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c6f417c29b464762944a721d477b2b13; no voters: 61c27cf3d4c846748ae6752e0fa652b9, 8970a89e5fd744008240ecd668272a4a
I20260811 01:31:10.584168 8372 raft_consensus.cc:2726] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260811 01:31:11.335585 8399 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:11.335770 8399 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:11.336442 8399 leader_election.cc:288] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755), 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427)
W20260811 01:31:11.338641 8229 leader_election.cc:334] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755): Network error: Client connection negotiation failed: client connection to 127.5.107.254:33755: connect: Connection refused (error 111)
W20260811 01:31:11.338814 8231 leader_election.cc:334] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
I20260811 01:31:11.338979 8231 leader_election.cc:302] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c6f417c29b464762944a721d477b2b13; no voters: 61c27cf3d4c846748ae6752e0fa652b9, 8970a89e5fd744008240ecd668272a4a
I20260811 01:31:11.339257 8399 raft_consensus.cc:2726] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260811 01:31:12.433926 8437 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:12.434247 8437 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:12.435004 8437 leader_election.cc:288] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755), 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427)
W20260811 01:31:12.437433 8229 leader_election.cc:334] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755): Network error: Client connection negotiation failed: client connection to 127.5.107.254:33755: connect: Connection refused (error 111)
W20260811 01:31:12.437615 8231 leader_election.cc:334] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
I20260811 01:31:12.437796 8231 leader_election.cc:302] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c6f417c29b464762944a721d477b2b13; no voters: 61c27cf3d4c846748ae6752e0fa652b9, 8970a89e5fd744008240ecd668272a4a
I20260811 01:31:12.438050 8437 raft_consensus.cc:2726] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260811 01:31:13.043871 5551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZLXb2N/build/asan/bin/kudu
/tmp/dist-test-taskZLXb2N/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.107.253:36427
--webserver_interface=127.5.107.253
--webserver_port=42225
--builtin_ntp_servers=127.5.107.212:36421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519
--hive_metastore_uris=thrift://127.0.0.1:36237
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260811 01:31:13.400760 8461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:31:13.401160 8461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:31:13.401309 8461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:31:13.409189 8461 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:31:13.409365 8461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:31:13.409471 8461 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:31:13.409574 8461 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:31:13.419616 8461 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.107.212:36421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:36237
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.107.253:36427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.107.253
--webserver_port=42225
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:31:13.421157 8461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:31:13.423195 8461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:31:13.435750 8476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:31:13.435834 8478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:31:13.435807 8474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:31:13.437076 8461 server_base.cc:1034] running on GCE node
I20260811 01:31:13.437932 8461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:31:13.439499 8461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:31:13.440744 8461 hybrid_clock.cc:648] HybridClock initialized: now 1786411873440646 us; error 61 us; skew 500 ppm
I20260811 01:31:13.441104 8461 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:31:13.449131 8461 webserver.cc:460] Webserver started at http://127.5.107.253:42225/ using document root <none> and password file <none>
I20260811 01:31:13.449934 8461 fs_manager.cc:359] Metadata directory not provided
I20260811 01:31:13.450170 8461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:31:13.455680 8461 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 01:31:13.458914 8488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:31:13.460346 8461 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 01:31:13.460561 8461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/data,/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/wal
uuid: "8970a89e5fd744008240ecd668272a4a"
format_stamp: "Formatted at 2026-08-11 01:30:53 on dist-test-slave-2p8l"
I20260811 01:31:13.462105 8461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:31:13.491401 8461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:31:13.492376 8461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:31:13.492692 8461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:31:13.513154 8461 rpc_server.cc:307] RPC server started. Bound to: 127.5.107.253:36427
I20260811 01:31:13.513173 8587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.107.253:36427 every 8 connection(s)
I20260811 01:31:13.514782 8461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/data/info.pb
I20260811 01:31:13.521638 8589 sys_catalog.cc:263] Verifying existing consensus state
I20260811 01:31:13.523047 5551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZLXb2N/build/asan/bin/kudu as pid 8461
I20260811 01:31:13.525924 8589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a: Bootstrap starting.
I20260811 01:31:13.537818 8589 log.cc:824] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a: Log is configured to *not* fsync() on all Append() calls
I20260811 01:31:13.556493 8589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=10 ignored=0} mutations{seen=7 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 01:31:13.557092 8589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a: Bootstrap complete.
I20260811 01:31:13.565734 6658 heartbeater.cc:347] Connected to a master server at 127.5.107.253:36427
I20260811 01:31:13.568091 8589 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:13.569362 8589 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8970a89e5fd744008240ecd668272a4a, State: Initialized, Role: FOLLOWER
I20260811 01:31:13.569986 8589 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:13.570879 8593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } } }
I20260811 01:31:13.571234 8593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:31:13.571961 8589 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:31:13.585217 8614 catalog_manager.cc:1259] Loaded cluster ID: 2f0039b0f9ec4f479f59cf7d8b74b7a2
I20260811 01:31:13.585429 8614 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a: loading cluster ID for follower catalog manager: success
I20260811 01:31:13.590018 8614 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a: acquiring CA information for follower catalog manager: success
I20260811 01:31:13.592886 8614 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 01:31:13.596911 8461 master_runner.cc:186] Error getting master registration for 127.5.107.254:33755: Network error: Client connection negotiation failed: client connection to 127.5.107.254:33755: connect: Connection refused (error 111),
I20260811 01:31:13.606032 8461 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:31:13.867520 6896 heartbeater.cc:347] Connected to a master server at 127.5.107.253:36427
I20260811 01:31:13.871029 8519 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" instance_seqno: 1786411855798958) as {username='slave'} at 127.5.107.194:39105; Asking this server to re-register.
I20260811 01:31:13.871982 6896 heartbeater.cc:464] Registering TS with master...
I20260811 01:31:13.874874 8518 ts_manager.cc:194] Registered new tserver with Master: 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971)
I20260811 01:31:13.974777 7113 heartbeater.cc:347] Connected to a master server at 127.5.107.253:36427
I20260811 01:31:13.976771 8518 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" instance_seqno: 1786411856431448) as {username='slave'} at 127.5.107.195:51415; Asking this server to re-register.
I20260811 01:31:13.977429 7113 heartbeater.cc:464] Registering TS with master...
I20260811 01:31:13.979017 8518 ts_manager.cc:194] Registered new tserver with Master: 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
I20260811 01:31:14.070451 6658 heartbeater.cc:347] Connected to a master server at 127.5.107.253:36427
I20260811 01:31:14.072165 8518 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "4a331de1488e4a0986b490917efea000" instance_seqno: 1786411855240795) as {username='slave'} at 127.5.107.193:35345; Asking this server to re-register.
I20260811 01:31:14.072909 6658 heartbeater.cc:464] Registering TS with master...
I20260811 01:31:14.074554 8518 ts_manager.cc:194] Registered new tserver with Master: 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
I20260811 01:31:14.165724 8638 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:14.166018 8638 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:14.167938 8638 leader_election.cc:288] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:31:14.168977 8495 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.107.254:33755: connect: Connection refused (error 111)
W20260811 01:31:14.171262 8495 leader_election.cc:334] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755): Network error: Client connection negotiation failed: client connection to 127.5.107.254:33755: connect: Connection refused (error 111)
I20260811 01:31:14.175706 8269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8970a89e5fd744008240ecd668272a4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: false dest_uuid: "c6f417c29b464762944a721d477b2b13" is_pre_election: true
I20260811 01:31:14.176322 8269 raft_consensus.cc:2379] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8970a89e5fd744008240ecd668272a4a in current term 3: Already voted for candidate 61c27cf3d4c846748ae6752e0fa652b9 in this term.
I20260811 01:31:14.177392 8496 leader_election.cc:302] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8970a89e5fd744008240ecd668272a4a; no voters: 61c27cf3d4c846748ae6752e0fa652b9, c6f417c29b464762944a721d477b2b13
I20260811 01:31:14.177946 8638 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 2 FOLLOWER]: Advancing to term 3
I20260811 01:31:14.178138 8650 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:14.178324 8650 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:14.179071 8650 leader_election.cc:288] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755), 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427)
I20260811 01:31:14.184260 8638 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260811 01:31:14.184212 8229 leader_election.cc:334] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755): Network error: Client connection negotiation failed: client connection to 127.5.107.254:33755: connect: Connection refused (error 111)
I20260811 01:31:14.187428 8538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c6f417c29b464762944a721d477b2b13" candidate_term: 4 candidate_status { last_received { term: 3 index: 19 } } ignore_live_leader: false dest_uuid: "8970a89e5fd744008240ecd668272a4a" is_pre_election: true
I20260811 01:31:14.187908 8538 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6f417c29b464762944a721d477b2b13 in term 3.
I20260811 01:31:14.188544 8231 leader_election.cc:302] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13; no voters: 61c27cf3d4c846748ae6752e0fa652b9
I20260811 01:31:14.188854 8650 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260811 01:31:14.188998 8650 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:31:14.189097 8650 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 3 FOLLOWER]: Advancing to term 4
I20260811 01:31:14.192991 8650 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:14.193660 8650 leader_election.cc:288] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 election: Requested vote from peers 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755), 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427)
I20260811 01:31:14.194458 8538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c6f417c29b464762944a721d477b2b13" candidate_term: 4 candidate_status { last_received { term: 3 index: 19 } } ignore_live_leader: false dest_uuid: "8970a89e5fd744008240ecd668272a4a"
I20260811 01:31:14.194710 8538 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 3 FOLLOWER]: Advancing to term 4
W20260811 01:31:14.195291 8229 leader_election.cc:334] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755): Network error: Client connection negotiation failed: client connection to 127.5.107.254:33755: connect: Connection refused (error 111)
I20260811 01:31:14.196943 8538 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6f417c29b464762944a721d477b2b13 in term 4.
I20260811 01:31:14.197458 8231 leader_election.cc:302] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13; no voters: 61c27cf3d4c846748ae6752e0fa652b9
I20260811 01:31:14.197781 8650 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 4 FOLLOWER]: Leader election won for term 4
I20260811 01:31:14.198338 8650 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [term 4 LEADER]: Becoming Leader. State: Replica: c6f417c29b464762944a721d477b2b13, State: Running, Role: LEADER
I20260811 01:31:14.198875 8650 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 3.19, Last appended by leader: 19, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:14.201687 8658 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6f417c29b464762944a721d477b2b13. Latest consensus state: current_term: 4 leader_uuid: "c6f417c29b464762944a721d477b2b13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } } }
I20260811 01:31:14.202123 8658 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [sys.catalog]: This master's current role is: LEADER
I20260811 01:31:14.202765 8661 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:31:14.205396 8661 catalog_manager.cc:669] Loaded metadata for table default.table_f7f5b09418c242d092b1948b866994e5 [id=69f9a5a340894b4bb41b150af86e0d8a]
I20260811 01:31:14.206009 8661 catalog_manager.cc:669] Loaded metadata for table default.table_b6731279f2fb4ef2a0a845c55840c58b [id=89b85d47429d40c787b453da0787febc]
I20260811 01:31:14.206436 8661 catalog_manager.cc:669] Loaded metadata for table default.table_c93618b4070e4974a6a794087d73fa73 [id=a65d01fd84fa41a3b31bf35b60f5cbaf]
I20260811 01:31:14.208649 8661 tablet_loader.cc:96] loaded metadata for tablet 1cb26d431b24475a8cf62325267b0a3d (table default.table_c93618b4070e4974a6a794087d73fa73 [id=a65d01fd84fa41a3b31bf35b60f5cbaf])
I20260811 01:31:14.209034 8661 tablet_loader.cc:96] loaded metadata for tablet 24df17b4b73c4838aa887fe2c1d4bcd0 (table default.table_b6731279f2fb4ef2a0a845c55840c58b [id=89b85d47429d40c787b453da0787febc])
I20260811 01:31:14.209669 8661 tablet_loader.cc:96] loaded metadata for tablet a6bc3c3d5d904830bfda3966e47305da (table default.table_f7f5b09418c242d092b1948b866994e5 [id=69f9a5a340894b4bb41b150af86e0d8a])
I20260811 01:31:14.210219 8661 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:31:14.211232 8661 catalog_manager.cc:1259] Loaded cluster ID: 2f0039b0f9ec4f479f59cf7d8b74b7a2
I20260811 01:31:14.211340 8661 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:31:14.212651 8661 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:31:14.213632 8661 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13: Loaded TSK: 0
I20260811 01:31:14.214617 8661 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 01:31:14.215543 8661 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260811 01:31:14.216406 8661 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260811 01:31:14.381402 8253 master_service.cc:946] ReplaceTablet: received request to replace tablet a6bc3c3d5d904830bfda3966e47305da from {username='slave'} at 127.0.0.1:56224
I20260811 01:31:14.381939 8249 master_service.cc:946] ReplaceTablet: received request to replace tablet 1cb26d431b24475a8cf62325267b0a3d from {username='slave'} at 127.0.0.1:56224
I20260811 01:31:14.382123 8248 master_service.cc:946] ReplaceTablet: received request to replace tablet a6bc3c3d5d904830bfda3966e47305da from {username='slave'} at 127.0.0.1:56224
I20260811 01:31:14.382467 8247 master_service.cc:946] ReplaceTablet: received request to replace tablet a6bc3c3d5d904830bfda3966e47305da from {username='slave'} at 127.0.0.1:56224
I20260811 01:31:14.381662 8250 master_service.cc:946] ReplaceTablet: received request to replace tablet 1cb26d431b24475a8cf62325267b0a3d from {username='slave'} at 127.0.0.1:56224
I20260811 01:31:14.383737 8245 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:56224:
table { table_name: "default.Table_f7f5b09418c242d092b1948b866994e5" } new_table_name: "default.Table_414082fd726f49d4a5926d9c9c739495" modify_external_catalogs: true
I20260811 01:31:14.383450 8244 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56224:
name: "default.Table_977e759aa4644f0091ecc8c4613cffc2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:31:14.383584 8246 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56224:
name: "default.Table_9f9bb4e0b22447c79a5597bc7db4eec8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:31:14.384869 8241 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56224:
name: "default.Table_0bf1f8de8ab54285b6291b6c6e30b647"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:31:14.386194 8243 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56224:
name: "default.Table_5e80cfaf385a44328ab0113196274005"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:31:14.391625 8538 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 4 FOLLOWER]: Refusing update from remote peer c6f417c29b464762944a721d477b2b13: Log matching property violated. Preceding OpId in replica: term: 2 index: 14. Preceding OpId from leader: term: 4 index: 21. (index mismatch)
F20260811 01:31:14.392014 8341 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_d472cfe99d004f68bc6beda9b7c20bc4
*** Check failure stack trace: ***
I20260811 01:31:14.392795 8664 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 14, Time since last communication: 0.000s
W20260811 01:31:14.393640 8229 consensus_peers.cc:487] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 -> Peer 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755): Couldn't send request to peer 61c27cf3d4c846748ae6752e0fa652b9. Status: Network error: Client connection negotiation failed: client connection to 127.5.107.254:33755: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
@ 0x7f76ad5d89ad google::LogMessage::Fail() at ??:0
@ 0x7f76ad5db88a google::LogMessage::SendToLog() at ??:0
@ 0x7f76ad5d84de google::LogMessage::Flush() at ??:0
@ 0x7f76ad5d86d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7f76c3176ba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260811 01:31:14.412849 8667 mvcc.cc:205] Tried to move back new op lower bound from 7317142977663328256 to 7317142974597046272. Current Snapshot: MvccSnapshot[applied={T|T < 7317142977663328256 or (T in {7317142977663328256})}]
I20260811 01:31:14.413677 8638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6f417c29b464762944a721d477b2b13. Latest consensus state: current_term: 4 leader_uuid: "c6f417c29b464762944a721d477b2b13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } } }
I20260811 01:31:14.414189 8638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:31:14.414702 8669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "c6f417c29b464762944a721d477b2b13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } } }
I20260811 01:31:14.415283 8669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:31:14.415536 8669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "c6f417c29b464762944a721d477b2b13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } } }
I20260811 01:31:14.415814 8669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [sys.catalog]: This master's current role is: FOLLOWER
@ 0x7f76c3175901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260811 01:31:14.419955 8249 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13: Sending DeleteTablet for 3 replicas of tablet 1cb26d431b24475a8cf62325267b0a3d
I20260811 01:31:14.420594 8253 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13: Sending DeleteTablet for 3 replicas of tablet a6bc3c3d5d904830bfda3966e47305da
I20260811 01:31:14.423062 8253 catalog_manager.cc:6565] ReplaceTablet: tablet a6bc3c3d5d904830bfda3966e47305da deleted and replaced by tablet 96e04578bb164dd48866d5925b000eff
@ 0x7f76c317c66c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260811 01:31:14.429813 8249 catalog_manager.cc:6565] ReplaceTablet: tablet 1cb26d431b24475a8cf62325267b0a3d deleted and replaced by tablet 01ab509f88494b5f998fd1705c5d0f59
I20260811 01:31:14.433835 8650 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "c6f417c29b464762944a721d477b2b13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } } }
I20260811 01:31:14.437702 8253 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56224:
name: "default.Table_4fad8fd88a6444b2a3dbc99d87835413"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:31:14.438434 8650 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6f417c29b464762944a721d477b2b13 [sys.catalog]: This master's current role is: LEADER
I20260811 01:31:14.433809 8249 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56224:
name: "default.Table_e6b7b1e823f84e859b8956dc8c474b29"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:31:14.436568 8250 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56224:
name: "default.Table_34b4086ff0f64632adbac74648d9874a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:31:14.436568 8248 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:56224:
table { table_name: "default.Table_b6731279f2fb4ef2a0a845c55840c58b" } new_table_name: "default.Table_ae0986f440b7487789e0721a121aa6d3" modify_external_catalogs: true
I20260811 01:31:14.436579 8247 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56224:
name: "default.Table_1354313d091b46d88f34c2ccd44bf8e0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
@ 0x7f76c317c4e9 std::_Function_handler<>::_M_invoke() at ??:0
I20260811 01:31:14.448350 6554 tablet_service.cc:1514] Processing DeleteTablet for tablet a6bc3c3d5d904830bfda3966e47305da with delete_type TABLET_DATA_DELETED (replaced by tablet 96e04578bb164dd48866d5925b000eff) from {username='slave'} at 127.0.0.1:37338
I20260811 01:31:14.448518 6557 tablet_service.cc:1514] Processing DeleteTablet for tablet 1cb26d431b24475a8cf62325267b0a3d with delete_type TABLET_DATA_DELETED (replaced by tablet 01ab509f88494b5f998fd1705c5d0f59) from {username='slave'} at 127.0.0.1:37338
@ 0x9edb7d std::function<>::operator()() at ??:?
I20260811 01:31:14.456602 7007 tablet_service.cc:1514] Processing DeleteTablet for tablet 1cb26d431b24475a8cf62325267b0a3d with delete_type TABLET_DATA_DELETED (replaced by tablet 01ab509f88494b5f998fd1705c5d0f59) from {username='slave'} at 127.0.0.1:45846
I20260811 01:31:14.457136 7006 tablet_service.cc:1514] Processing DeleteTablet for tablet a6bc3c3d5d904830bfda3966e47305da with delete_type TABLET_DATA_DELETED (replaced by tablet 96e04578bb164dd48866d5925b000eff) from {username='slave'} at 127.0.0.1:45846
@ 0x7f76b0202eaf kudu::Thread::SuperviseThread() at ??:0
I20260811 01:31:14.462508 7008 tablet_service.cc:1467] Processing CreateTablet for tablet 96e04578bb164dd48866d5925b000eff (DEFAULT_TABLE table=default.table_f7f5b09418c242d092b1948b866994e5 [id=69f9a5a340894b4bb41b150af86e0d8a]), partition=RANGE (key) PARTITION UNBOUNDED
@ 0x7f76b34dc6db start_thread at ??:0
I20260811 01:31:14.463115 7008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96e04578bb164dd48866d5925b000eff. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7f76ac47261f clone at ??:0
I20260811 01:31:14.466696 7006 tablet_service.cc:1467] Processing CreateTablet for tablet 01ab509f88494b5f998fd1705c5d0f59 (DEFAULT_TABLE table=default.table_c93618b4070e4974a6a794087d73fa73 [id=a65d01fd84fa41a3b31bf35b60f5cbaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:31:14.467196 7006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01ab509f88494b5f998fd1705c5d0f59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:31:14.479362 8694 tablet_bootstrap.cc:492] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af: Bootstrap starting.
I20260811 01:31:14.481700 8690 tablet_replica.cc:330] T a6bc3c3d5d904830bfda3966e47305da P 4a331de1488e4a0986b490917efea000: stopping tablet replica
I20260811 01:31:14.481923 8690 raft_consensus.cc:2229] T a6bc3c3d5d904830bfda3966e47305da P 4a331de1488e4a0986b490917efea000 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:31:14.482407 8690 raft_consensus.cc:2258] T a6bc3c3d5d904830bfda3966e47305da P 4a331de1488e4a0986b490917efea000 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:31:14.482595 8694 tablet_bootstrap.cc:654] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af: Neither blocks nor log segments found. Creating new log.
I20260811 01:31:14.483068 8690 ts_tablet_manager.cc:1905] T a6bc3c3d5d904830bfda3966e47305da P 4a331de1488e4a0986b490917efea000: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:31:14.488119 8691 tablet_replica.cc:330] T 1cb26d431b24475a8cf62325267b0a3d P 992432fc73c94fcc9f2142b27ef725af: stopping tablet replica
I20260811 01:31:14.488379 8691 raft_consensus.cc:2229] T 1cb26d431b24475a8cf62325267b0a3d P 992432fc73c94fcc9f2142b27ef725af [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:31:14.489073 8691 raft_consensus.cc:2258] T 1cb26d431b24475a8cf62325267b0a3d P 992432fc73c94fcc9f2142b27ef725af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:31:14.489215 8694 tablet_bootstrap.cc:492] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af: No bootstrap required, opened a new log
I20260811 01:31:14.489391 8694 ts_tablet_manager.cc:1397] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.000s
I20260811 01:31:14.489760 8691 ts_tablet_manager.cc:1905] T 1cb26d431b24475a8cf62325267b0a3d P 992432fc73c94fcc9f2142b27ef725af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:31:14.490474 8694 raft_consensus.cc:348] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:31:14.490833 8694 raft_consensus.cc:374] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:31:14.490962 8694 raft_consensus.cc:729] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 992432fc73c94fcc9f2142b27ef725af, State: Initialized, Role: FOLLOWER
I20260811 01:31:14.491230 8694 consensus_queue.cc:260] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:31:14.495687 8690 ts_tablet_manager.cc:1918] T a6bc3c3d5d904830bfda3966e47305da P 4a331de1488e4a0986b490917efea000: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 01:31:14.495831 8690 log.cc:1196] T a6bc3c3d5d904830bfda3966e47305da P 4a331de1488e4a0986b490917efea000: Deleting WAL directory at /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-0/wal/wals/a6bc3c3d5d904830bfda3966e47305da
I20260811 01:31:14.496326 8690 ts_tablet_manager.cc:1939] T a6bc3c3d5d904830bfda3966e47305da P 4a331de1488e4a0986b490917efea000: Deleting consensus metadata
I20260811 01:31:14.496471 8694 ts_tablet_manager.cc:1428] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260811 01:31:14.500344 8694 tablet_bootstrap.cc:492] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af: Bootstrap starting.
I20260811 01:31:14.502676 8694 tablet_bootstrap.cc:654] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af: Neither blocks nor log segments found. Creating new log.
I20260811 01:31:14.503329 8691 ts_tablet_manager.cc:1918] T 1cb26d431b24475a8cf62325267b0a3d P 992432fc73c94fcc9f2142b27ef725af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 01:31:14.503477 8691 log.cc:1196] T 1cb26d431b24475a8cf62325267b0a3d P 992432fc73c94fcc9f2142b27ef725af: Deleting WAL directory at /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-2/wal/wals/1cb26d431b24475a8cf62325267b0a3d
I20260811 01:31:14.503942 8691 ts_tablet_manager.cc:1939] T 1cb26d431b24475a8cf62325267b0a3d P 992432fc73c94fcc9f2142b27ef725af: Deleting consensus metadata
I20260811 01:31:14.505793 8690 tablet_replica.cc:330] T 1cb26d431b24475a8cf62325267b0a3d P 4a331de1488e4a0986b490917efea000: stopping tablet replica
I20260811 01:31:14.506018 8690 raft_consensus.cc:2229] T 1cb26d431b24475a8cf62325267b0a3d P 4a331de1488e4a0986b490917efea000 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:31:14.506245 8690 raft_consensus.cc:2258] T 1cb26d431b24475a8cf62325267b0a3d P 4a331de1488e4a0986b490917efea000 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:31:14.506673 8691 tablet_replica.cc:330] T a6bc3c3d5d904830bfda3966e47305da P 992432fc73c94fcc9f2142b27ef725af: stopping tablet replica
I20260811 01:31:14.506927 8690 ts_tablet_manager.cc:1905] T 1cb26d431b24475a8cf62325267b0a3d P 4a331de1488e4a0986b490917efea000: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:31:14.506938 8691 raft_consensus.cc:2229] T a6bc3c3d5d904830bfda3966e47305da P 992432fc73c94fcc9f2142b27ef725af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:31:14.507145 8691 raft_consensus.cc:2258] T a6bc3c3d5d904830bfda3966e47305da P 992432fc73c94fcc9f2142b27ef725af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:31:14.507575 8694 tablet_bootstrap.cc:492] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af: No bootstrap required, opened a new log
I20260811 01:31:14.507752 8694 ts_tablet_manager.cc:1397] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 01:31:14.507808 8691 ts_tablet_manager.cc:1905] T a6bc3c3d5d904830bfda3966e47305da P 992432fc73c94fcc9f2142b27ef725af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:31:14.508667 8694 raft_consensus.cc:348] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } }
I20260811 01:31:14.508947 8694 raft_consensus.cc:374] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:31:14.509029 8694 raft_consensus.cc:729] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 992432fc73c94fcc9f2142b27ef725af, State: Initialized, Role: FOLLOWER
I20260811 01:31:14.509472 8694 consensus_queue.cc:260] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } }
I20260811 01:31:14.513777 8690 ts_tablet_manager.cc:1918] T 1cb26d431b24475a8cf62325267b0a3d P 4a331de1488e4a0986b490917efea000: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 01:31:14.513937 8690 log.cc:1196] T 1cb26d431b24475a8cf62325267b0a3d P 4a331de1488e4a0986b490917efea000: Deleting WAL directory at /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-0/wal/wals/1cb26d431b24475a8cf62325267b0a3d
I20260811 01:31:14.514112 8694 ts_tablet_manager.cc:1428] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260811 01:31:14.514395 8690 ts_tablet_manager.cc:1939] T 1cb26d431b24475a8cf62325267b0a3d P 4a331de1488e4a0986b490917efea000: Deleting consensus metadata
I20260811 01:31:14.519845 8691 ts_tablet_manager.cc:1918] T a6bc3c3d5d904830bfda3966e47305da P 992432fc73c94fcc9f2142b27ef725af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 01:31:14.519989 8691 log.cc:1196] T a6bc3c3d5d904830bfda3966e47305da P 992432fc73c94fcc9f2142b27ef725af: Deleting WAL directory at /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/ts-2/wal/wals/a6bc3c3d5d904830bfda3966e47305da
I20260811 01:31:14.520313 8691 ts_tablet_manager.cc:1939] T a6bc3c3d5d904830bfda3966e47305da P 992432fc73c94fcc9f2142b27ef725af: Deleting consensus metadata
W20260811 01:31:14.682191 6469 connection.cc:535] server connection from 127.0.0.1:37338 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 01:31:14.682191 6921 connection.cc:535] server connection from 127.0.0.1:45846 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 01:31:14.682775 6923 connection.cc:535] client connection to 127.5.107.252:45519 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 01:31:14.682906 8686 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:31:14.424587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36232 (local address 127.5.107.194:32971)
0811 01:31:14.433624 (+ 9037us) server_negotiation.cc:206] Beginning negotiation
0811 01:31:14.433630 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:31:14.452012 (+ 18382us) server_negotiation.cc:407] Connection header received
0811 01:31:14.452216 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:31:14.452225 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:31:14.452329 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:31:14.452445 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:31:14.453265 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:31:14.454678 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:31:14.682376 (+227698us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36232: BlockingRecv error: recv got EOF from 127.0.0.1:36232 (error 108)
Metrics: {"server-negotiator.queue_time_us":8881,"thread_start_us":8757,"threads_started":1}
W20260811 01:31:14.683154 6471 connection.cc:535] client connection to 127.5.107.252:45519 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 01:31:14.683768 7131 connection.cc:535] client connection to 127.5.107.252:45519 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 01:31:14.685175 7131 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.5.107.252:45519)
W20260811 01:31:14.685834 7131 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.5.107.252:45519)
W20260811 01:31:14.686007 7131 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.5.107.252:45519)
I20260811 01:31:14.959198 8709 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c6f417c29b464762944a721d477b2b13)
I20260811 01:31:14.959390 8709 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:14.960124 8709 leader_election.cc:288] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:31:14.962374 8495 leader_election.cc:334] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755): Network error: Client connection negotiation failed: client connection to 127.5.107.254:33755: connect: Connection refused (error 111)
W20260811 01:31:14.962616 8496 leader_election.cc:334] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:31:14.962785 8496 leader_election.cc:302] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8970a89e5fd744008240ecd668272a4a; no voters: 61c27cf3d4c846748ae6752e0fa652b9, c6f417c29b464762944a721d477b2b13
I20260811 01:31:14.963073 8709 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260811 01:31:15.295514 8716 raft_consensus.cc:482] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:15.295728 8716 raft_consensus.cc:504] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } }
I20260811 01:31:15.296353 8716 leader_election.cc:288] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:31:15.297596 6920 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
W20260811 01:31:15.297876 6923 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
I20260811 01:31:15.298062 6923 leader_election.cc:302] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:31:15.298390 8716 raft_consensus.cc:2726] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:31:15.423545 8716 raft_consensus.cc:482] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:15.423790 8716 raft_consensus.cc:504] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:31:15.424743 8716 leader_election.cc:288] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:31:15.425953 6920 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
W20260811 01:31:15.426194 6923 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
I20260811 01:31:15.426375 6923 leader_election.cc:302] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:31:15.426667 8716 raft_consensus.cc:2726] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:31:15.464705 8717 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c6f417c29b464762944a721d477b2b13)
I20260811 01:31:15.464905 8717 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:15.465559 8717 leader_election.cc:288] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:31:15.467762 8496 leader_election.cc:334] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
W20260811 01:31:15.467947 8495 leader_election.cc:334] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755): Network error: Client connection negotiation failed: client connection to 127.5.107.254:33755: connect: Connection refused (error 111)
I20260811 01:31:15.468122 8495 leader_election.cc:302] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8970a89e5fd744008240ecd668272a4a; no voters: 61c27cf3d4c846748ae6752e0fa652b9, c6f417c29b464762944a721d477b2b13
I20260811 01:31:15.468389 8717 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260811 01:31:16.240486 8727 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c6f417c29b464762944a721d477b2b13)
I20260811 01:31:16.240671 8727 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:16.241304 8727 leader_election.cc:288] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:31:16.243662 8496 leader_election.cc:334] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
W20260811 01:31:16.243918 8495 leader_election.cc:334] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755): Network error: Client connection negotiation failed: client connection to 127.5.107.254:33755: connect: Connection refused (error 111)
I20260811 01:31:16.244065 8495 leader_election.cc:302] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8970a89e5fd744008240ecd668272a4a; no voters: 61c27cf3d4c846748ae6752e0fa652b9, c6f417c29b464762944a721d477b2b13
I20260811 01:31:16.244369 8727 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260811 01:31:16.615191 5551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZLXb2N/build/asan/bin/kudu
/tmp/dist-test-taskZLXb2N/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.107.254:33755
--webserver_interface=127.5.107.254
--webserver_port=45627
--builtin_ntp_servers=127.5.107.212:36421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519
--hive_metastore_uris=thrift://127.0.0.1:36237
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260811 01:31:16.719039 8736 raft_consensus.cc:482] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 992432fc73c94fcc9f2142b27ef725af)
I20260811 01:31:16.719296 8736 raft_consensus.cc:504] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:31:16.720193 8736 leader_election.cc:288] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:31:16.721305 6702 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
W20260811 01:31:16.721570 6699 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
I20260811 01:31:16.721765 6699 leader_election.cc:302] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:31:16.722119 8736 raft_consensus.cc:2726] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:31:16.929769 8736 raft_consensus.cc:482] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4a331de1488e4a0986b490917efea000)
I20260811 01:31:16.930043 8736 raft_consensus.cc:504] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:31:16.930807 8736 leader_election.cc:288] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:31:16.931871 6702 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
W20260811 01:31:16.932061 6699 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
I20260811 01:31:16.932263 6699 leader_election.cc:302] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:31:16.932582 8736 raft_consensus.cc:2726] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260811 01:31:16.984073 8735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:31:16.984427 8735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:31:16.984591 8735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:31:16.994822 8735 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:31:16.995016 8735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:31:16.995110 8735 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:31:16.995216 8735 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:31:17.006286 8735 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.107.212:36421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:36237
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.107.254:33755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.107.254
--webserver_port=45627
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:31:17.007791 8735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:31:17.009691 8735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:31:17.023795 8746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:31:17.024585 8748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:31:17.024715 8744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:31:17.025417 8735 server_base.cc:1034] running on GCE node
I20260811 01:31:17.026525 8735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:31:17.028484 8735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:31:17.029780 8735 hybrid_clock.cc:648] HybridClock initialized: now 1786411877029707 us; error 33 us; skew 500 ppm
I20260811 01:31:17.030210 8735 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:31:17.033289 8735 webserver.cc:460] Webserver started at http://127.5.107.254:45627/ using document root <none> and password file <none>
I20260811 01:31:17.034309 8735 fs_manager.cc:359] Metadata directory not provided
I20260811 01:31:17.034485 8735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:31:17.042472 8735 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.004s
I20260811 01:31:17.046821 8761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:31:17.051402 8735 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.002s sys 0.000s
I20260811 01:31:17.051636 8735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/data,/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/wal
uuid: "61c27cf3d4c846748ae6752e0fa652b9"
format_stamp: "Formatted at 2026-08-11 01:30:53 on dist-test-slave-2p8l"
I20260811 01:31:17.052963 8735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:31:17.079483 8735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:31:17.080801 8735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:31:17.081245 8735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:31:17.104929 8735 rpc_server.cc:307] RPC server started. Bound to: 127.5.107.254:33755
I20260811 01:31:17.104974 8842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.107.254:33755 every 8 connection(s)
I20260811 01:31:17.106739 8735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/data/info.pb
I20260811 01:31:17.113842 8844 sys_catalog.cc:263] Verifying existing consensus state
I20260811 01:31:17.116900 5551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZLXb2N/build/asan/bin/kudu as pid 8735
I20260811 01:31:17.118487 8844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9: Bootstrap starting.
I20260811 01:31:17.133143 8844 log.cc:824] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9: Log is configured to *not* fsync() on all Append() calls
I20260811 01:31:17.151532 8844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=19 overwritten=0 applied=19 ignored=0} inserts{seen=12 ignored=0} mutations{seen=11 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 01:31:17.152146 8844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9: Bootstrap complete.
I20260811 01:31:17.159953 8844 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:17.160899 8844 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61c27cf3d4c846748ae6752e0fa652b9, State: Initialized, Role: FOLLOWER
I20260811 01:31:17.161602 8844 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 19, Last appended: 3.19, Last appended by leader: 19, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:17.162554 8850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } } }
I20260811 01:31:17.162870 8850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:31:17.163673 8844 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:31:17.183866 8875 catalog_manager.cc:1259] Loaded cluster ID: 2f0039b0f9ec4f479f59cf7d8b74b7a2
I20260811 01:31:17.184119 8875 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9: loading cluster ID for follower catalog manager: success
I20260811 01:31:17.189713 8875 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9: acquiring CA information for follower catalog manager: success
I20260811 01:31:17.193665 8875 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 01:31:17.203536 8735 master_runner.cc:186] Error getting master registration for 127.5.107.252:45519: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111),
I20260811 01:31:17.210907 8735 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:31:17.378450 6891 heartbeater.cc:347] Connected to a master server at 127.5.107.254:33755
I20260811 01:31:17.381489 8789 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" instance_seqno: 1786411855798958) as {username='slave'} at 127.5.107.194:56443; Asking this server to re-register.
I20260811 01:31:17.382274 6891 heartbeater.cc:464] Registering TS with master...
I20260811 01:31:17.384102 8789 ts_manager.cc:194] Registered new tserver with Master: 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971)
I20260811 01:31:17.529569 7111 heartbeater.cc:347] Connected to a master server at 127.5.107.254:33755
I20260811 01:31:17.531139 8789 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" instance_seqno: 1786411856431448) as {username='slave'} at 127.5.107.195:53365; Asking this server to re-register.
I20260811 01:31:17.531728 7111 heartbeater.cc:464] Registering TS with master...
I20260811 01:31:17.533052 8789 ts_manager.cc:194] Registered new tserver with Master: 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
I20260811 01:31:17.572177 6657 heartbeater.cc:347] Connected to a master server at 127.5.107.254:33755
I20260811 01:31:17.573527 8789 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "4a331de1488e4a0986b490917efea000" instance_seqno: 1786411855240795) as {username='slave'} at 127.5.107.193:49667; Asking this server to re-register.
I20260811 01:31:17.574427 6657 heartbeater.cc:464] Registering TS with master...
I20260811 01:31:17.575817 8789 ts_manager.cc:194] Registered new tserver with Master: 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
I20260811 01:31:17.601531 8850 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:17.601923 8850 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:17.604190 8850 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
I20260811 01:31:17.605046 8538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "61c27cf3d4c846748ae6752e0fa652b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 19 } } ignore_live_leader: false dest_uuid: "8970a89e5fd744008240ecd668272a4a" is_pre_election: true
W20260811 01:31:17.605297 8765 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:31:17.605466 8538 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 61c27cf3d4c846748ae6752e0fa652b9 in current term 4: Already voted for candidate c6f417c29b464762944a721d477b2b13 in this term.
W20260811 01:31:17.607080 8765 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:31:17.607261 8765 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:31:17.607656 8850 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 3 FOLLOWER]: Advancing to term 4
I20260811 01:31:17.612316 8850 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260811 01:31:17.765125 8885 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c6f417c29b464762944a721d477b2b13)
I20260811 01:31:17.765424 8885 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:17.766459 8885 leader_election.cc:288] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:31:17.773451 8496 leader_election.cc:334] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:31:17.773557 8804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8970a89e5fd744008240ecd668272a4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 23 } } ignore_live_leader: false dest_uuid: "61c27cf3d4c846748ae6752e0fa652b9" is_pre_election: true
I20260811 01:31:17.774175 8804 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8970a89e5fd744008240ecd668272a4a in term 4.
I20260811 01:31:17.774744 8495 leader_election.cc:302] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9, 8970a89e5fd744008240ecd668272a4a; no voters: c6f417c29b464762944a721d477b2b13
I20260811 01:31:17.775137 8885 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260811 01:31:17.775305 8885 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 4 FOLLOWER]: Starting leader election (detected failure of leader c6f417c29b464762944a721d477b2b13)
I20260811 01:31:17.775434 8885 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 4 FOLLOWER]: Advancing to term 5
I20260811 01:31:17.777779 8885 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:17.778606 8885 leader_election.cc:288] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [CANDIDATE]: Term 5 election: Requested vote from peers 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
I20260811 01:31:17.779230 8804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8970a89e5fd744008240ecd668272a4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 23 } } ignore_live_leader: false dest_uuid: "61c27cf3d4c846748ae6752e0fa652b9"
I20260811 01:31:17.779491 8804 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 4 FOLLOWER]: Advancing to term 5
W20260811 01:31:17.780472 8496 leader_election.cc:334] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:31:17.782155 8804 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8970a89e5fd744008240ecd668272a4a in term 5.
I20260811 01:31:17.782845 8495 leader_election.cc:302] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9, 8970a89e5fd744008240ecd668272a4a; no voters: c6f417c29b464762944a721d477b2b13
I20260811 01:31:17.783202 8885 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 5 FOLLOWER]: Leader election won for term 5
I20260811 01:31:17.783677 8885 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 5 LEADER]: Becoming Leader. State: Replica: 8970a89e5fd744008240ecd668272a4a, State: Running, Role: LEADER
I20260811 01:31:17.784222 8885 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 4.23, Last appended by leader: 23, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:17.786280 8894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8970a89e5fd744008240ecd668272a4a. Latest consensus state: current_term: 5 leader_uuid: "8970a89e5fd744008240ecd668272a4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } } }
I20260811 01:31:17.786806 8894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [sys.catalog]: This master's current role is: LEADER
I20260811 01:31:17.787593 8897 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:31:17.790588 8897 catalog_manager.cc:669] Loaded metadata for table default.table_f7f5b09418c242d092b1948b866994e5 [id=69f9a5a340894b4bb41b150af86e0d8a]
I20260811 01:31:17.791131 8897 catalog_manager.cc:669] Loaded metadata for table default.table_b6731279f2fb4ef2a0a845c55840c58b [id=89b85d47429d40c787b453da0787febc]
I20260811 01:31:17.791563 8897 catalog_manager.cc:669] Loaded metadata for table default.table_c93618b4070e4974a6a794087d73fa73 [id=a65d01fd84fa41a3b31bf35b60f5cbaf]
I20260811 01:31:17.793362 8897 tablet_loader.cc:96] loaded metadata for tablet 01ab509f88494b5f998fd1705c5d0f59 (table default.table_c93618b4070e4974a6a794087d73fa73 [id=a65d01fd84fa41a3b31bf35b60f5cbaf])
I20260811 01:31:17.793969 8897 tablet_loader.cc:96] loaded metadata for tablet 24df17b4b73c4838aa887fe2c1d4bcd0 (table default.table_b6731279f2fb4ef2a0a845c55840c58b [id=89b85d47429d40c787b453da0787febc])
I20260811 01:31:17.794631 8897 tablet_loader.cc:96] loaded metadata for tablet 96e04578bb164dd48866d5925b000eff (table default.table_f7f5b09418c242d092b1948b866994e5 [id=69f9a5a340894b4bb41b150af86e0d8a])
I20260811 01:31:17.795282 8897 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:31:17.796317 8897 catalog_manager.cc:1259] Loaded cluster ID: 2f0039b0f9ec4f479f59cf7d8b74b7a2
I20260811 01:31:17.796440 8897 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:31:17.797824 8897 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:31:17.798924 8897 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a: Loaded TSK: 0
I20260811 01:31:17.799889 8897 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 01:31:17.800856 8897 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260811 01:31:17.801767 8897 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260811 01:31:17.810349 8518 master_service.cc:946] ReplaceTablet: received request to replace tablet 1cb26d431b24475a8cf62325267b0a3d from {username='slave'} at 127.0.0.1:41682
I20260811 01:31:17.810401 8519 master_service.cc:946] ReplaceTablet: received request to replace tablet 1cb26d431b24475a8cf62325267b0a3d from {username='slave'} at 127.0.0.1:41682
I20260811 01:31:17.810984 8515 master_service.cc:946] ReplaceTablet: received request to replace tablet a6bc3c3d5d904830bfda3966e47305da from {username='slave'} at 127.0.0.1:41682
I20260811 01:31:17.811069 8517 master_service.cc:946] ReplaceTablet: received request to replace tablet a6bc3c3d5d904830bfda3966e47305da from {username='slave'} at 127.0.0.1:41682
I20260811 01:31:17.812076 8513 master_service.cc:946] ReplaceTablet: received request to replace tablet 1cb26d431b24475a8cf62325267b0a3d from {username='slave'} at 127.0.0.1:41682
I20260811 01:31:17.813519 8504 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41682:
table { table_name: "default.Table_f7f5b09418c242d092b1948b866994e5" } new_table_name: "default.Table_414082fd726f49d4a5926d9c9c739495" modify_external_catalogs: true
I20260811 01:31:17.813757 8515 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41682:
name: "default.Table_5e80cfaf385a44328ab0113196274005"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:31:17.815295 8511 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41682:
table { table_name: "default.Table_c93618b4070e4974a6a794087d73fa73" } new_table_name: "default.Table_9be7f6a4237648ab8b421f62026d3ad5" modify_external_catalogs: true
I20260811 01:31:17.815409 8503 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41682:
table { table_name: "default.Table_b6731279f2fb4ef2a0a845c55840c58b" } new_table_name: "default.Table_ae0986f440b7487789e0721a121aa6d3" modify_external_catalogs: true
I20260811 01:31:17.821468 8519 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41682:
name: "default.Table_0bf1f8de8ab54285b6291b6c6e30b647"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:31:17.816287 8518 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41682:
name: "default.Table_977e759aa4644f0091ecc8c4613cffc2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:31:17.816208 8513 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41682:
name: "default.Table_34b4086ff0f64632adbac74648d9874a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:31:17.820971 8517 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41682:
name: "default.Table_1354313d091b46d88f34c2ccd44bf8e0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:31:17.815659 8505 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41682:
name: "default.Table_9f9bb4e0b22447c79a5597bc7db4eec8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:31:17.816781 8509 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41682:
name: "default.Table_16925b327c134e08987029291b57034c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260811 01:31:17.823156 8613 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_d472cfe99d004f68bc6beda9b7c20bc4
*** Check failure stack trace: ***
@ 0x7f5968cc19ad google::LogMessage::Fail() at ??:0
@ 0x7f5968cc488a google::LogMessage::SendToLog() at ??:0
@ 0x7f5968cc14de google::LogMessage::Flush() at ??:0
@ 0x7f5968cc16d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7f597e85fba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7f597e85e901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f597e86566c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7f597e8654e9 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f596b8ebeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f596ebc56db start_thread at ??:0
@ 0x7f5967b5b61f clone at ??:0
W20260811 01:31:18.040278 6923 connection.cc:535] client connection to 127.5.107.253:36427 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 01:31:18.040371 6702 connection.cc:535] client connection to 127.5.107.253:36427 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 01:31:18.040932 7131 connection.cc:535] client connection to 127.5.107.253:36427 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 01:31:18.041726 7131 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.5.107.253:36427)
W20260811 01:31:18.042520 7131 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.5.107.253:36427)
W20260811 01:31:18.042866 7131 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.5.107.253:36427)
I20260811 01:31:18.641021 8931 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:18.641294 8931 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:18.642190 8931 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:31:18.645570 8765 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:31:18.645857 8765 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:31:18.645952 8765 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:31:18.646341 8931 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260811 01:31:18.989246 8940 raft_consensus.cc:482] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:18.989538 8940 raft_consensus.cc:504] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:31:18.990484 8940 leader_election.cc:288] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:31:18.991701 6920 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
W20260811 01:31:18.991993 6923 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
I20260811 01:31:18.992146 6923 leader_election.cc:302] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:31:18.992467 8940 raft_consensus.cc:2726] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:31:19.293213 8941 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:19.293418 8941 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:19.294190 8941 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:31:19.299561 8765 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:31:19.299854 8765 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:31:19.300005 8765 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:31:19.300375 8941 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260811 01:31:19.648545 8950 raft_consensus.cc:482] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:19.648789 8950 raft_consensus.cc:504] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } }
I20260811 01:31:19.649470 8950 leader_election.cc:288] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:31:19.650319 6920 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
W20260811 01:31:19.650534 6923 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
I20260811 01:31:19.650688 6923 leader_election.cc:302] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:31:19.650950 8950 raft_consensus.cc:2726] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:31:19.912990 8953 raft_consensus.cc:482] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 992432fc73c94fcc9f2142b27ef725af)
I20260811 01:31:19.913210 8953 raft_consensus.cc:504] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:31:19.913879 8953 leader_election.cc:288] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:31:19.914933 6699 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
W20260811 01:31:19.915177 6702 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
I20260811 01:31:19.915345 6702 leader_election.cc:302] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:31:19.915633 8953 raft_consensus.cc:2726] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:31:20.412793 5551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZLXb2N/build/asan/bin/kudu
/tmp/dist-test-taskZLXb2N/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.107.253:36427
--webserver_interface=127.5.107.253
--webserver_port=42225
--builtin_ntp_servers=127.5.107.212:36421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519
--hive_metastore_uris=thrift://127.0.0.1:36237
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260811 01:31:20.602172 8955 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:20.602428 8955 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:20.603159 8955 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:31:20.606469 8765 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:31:20.606694 8765 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:31:20.606789 8765 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:31:20.607064 8955 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260811 01:31:20.760232 8954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:31:20.760562 8954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:31:20.760721 8954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:31:20.768234 8954 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:31:20.768358 8954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:31:20.768455 8954 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:31:20.768570 8954 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:31:20.777966 8954 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.107.212:36421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:36237
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.107.253:36427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.107.253
--webserver_port=42225
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:31:20.779256 8954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:31:20.780957 8954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:31:20.793675 8966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:31:20.793723 8967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:31:20.793779 8970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:31:20.794571 8954 server_base.cc:1034] running on GCE node
I20260811 01:31:20.795388 8954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:31:20.796774 8954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:31:20.798187 8954 hybrid_clock.cc:648] HybridClock initialized: now 1786411880798085 us; error 60 us; skew 500 ppm
I20260811 01:31:20.798580 8954 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:31:20.800953 8954 webserver.cc:460] Webserver started at http://127.5.107.253:42225/ using document root <none> and password file <none>
I20260811 01:31:20.801676 8954 fs_manager.cc:359] Metadata directory not provided
I20260811 01:31:20.801826 8954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:31:20.803570 8984 raft_consensus.cc:482] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4a331de1488e4a0986b490917efea000)
I20260811 01:31:20.803792 8984 raft_consensus.cc:504] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:31:20.804665 8984 leader_election.cc:288] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:31:20.805615 6699 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
W20260811 01:31:20.805967 6702 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
I20260811 01:31:20.806137 6702 leader_election.cc:302] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:31:20.806430 8984 raft_consensus.cc:2726] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:31:20.808305 8954 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260811 01:31:20.811259 8989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:31:20.812602 8954 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 01:31:20.812798 8954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/data,/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/wal
uuid: "8970a89e5fd744008240ecd668272a4a"
format_stamp: "Formatted at 2026-08-11 01:30:53 on dist-test-slave-2p8l"
I20260811 01:31:20.814316 8954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:31:20.837683 8954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:31:20.838559 8954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:31:20.838855 8954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:31:20.858112 8954 rpc_server.cc:307] RPC server started. Bound to: 127.5.107.253:36427
I20260811 01:31:20.858140 9078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.107.253:36427 every 8 connection(s)
I20260811 01:31:20.859823 8954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-1/data/info.pb
I20260811 01:31:20.864836 5551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZLXb2N/build/asan/bin/kudu as pid 8954
I20260811 01:31:20.867534 9079 sys_catalog.cc:263] Verifying existing consensus state
I20260811 01:31:20.872805 9079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a: Bootstrap starting.
I20260811 01:31:20.883659 9079 log.cc:824] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a: Log is configured to *not* fsync() on all Append() calls
I20260811 01:31:20.903964 9079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a: Bootstrap replayed 1/1 log segments. Stats: ops{read=24 overwritten=0 applied=23 ignored=0} inserts{seen=14 ignored=0} mutations{seen=15 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260811 01:31:20.904665 9079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a: Bootstrap complete.
I20260811 01:31:20.913559 9079 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 5 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:20.914767 9079 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 5 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8970a89e5fd744008240ecd668272a4a, State: Initialized, Role: FOLLOWER
I20260811 01:31:20.915383 9079 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 23, Last appended: 5.24, Last appended by leader: 24, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:20.916253 9086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 5 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } } }
I20260811 01:31:20.916621 9086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:31:20.917281 9079 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:31:20.931178 9108 catalog_manager.cc:1259] Loaded cluster ID: 2f0039b0f9ec4f479f59cf7d8b74b7a2
I20260811 01:31:20.931291 9108 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a: loading cluster ID for follower catalog manager: success
I20260811 01:31:20.935416 9108 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a: acquiring CA information for follower catalog manager: success
I20260811 01:31:20.938220 9108 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 01:31:20.965193 8954 master_runner.cc:186] Error getting master registration for 127.5.107.252:45519: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111),
I20260811 01:31:20.971443 8954 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:31:21.058864 6896 heartbeater.cc:347] Connected to a master server at 127.5.107.253:36427
I20260811 01:31:21.060137 7113 heartbeater.cc:347] Connected to a master server at 127.5.107.253:36427
I20260811 01:31:21.062431 9013 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" instance_seqno: 1786411856431448) as {username='slave'} at 127.5.107.195:48545; Asking this server to re-register.
I20260811 01:31:21.062463 9014 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" instance_seqno: 1786411855798958) as {username='slave'} at 127.5.107.194:58875; Asking this server to re-register.
I20260811 01:31:21.063186 7113 heartbeater.cc:464] Registering TS with master...
I20260811 01:31:21.063189 6896 heartbeater.cc:464] Registering TS with master...
I20260811 01:31:21.064765 9013 ts_manager.cc:194] Registered new tserver with Master: 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
I20260811 01:31:21.064862 9014 ts_manager.cc:194] Registered new tserver with Master: 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971)
I20260811 01:31:21.104990 6658 heartbeater.cc:347] Connected to a master server at 127.5.107.253:36427
I20260811 01:31:21.106149 9013 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "4a331de1488e4a0986b490917efea000" instance_seqno: 1786411855240795) as {username='slave'} at 127.5.107.193:58583; Asking this server to re-register.
I20260811 01:31:21.106864 6658 heartbeater.cc:464] Registering TS with master...
I20260811 01:31:21.108234 9013 ts_manager.cc:194] Registered new tserver with Master: 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
I20260811 01:31:21.487600 9116 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:21.487895 9116 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:21.489780 9116 leader_election.cc:288] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
I20260811 01:31:21.490480 8804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8970a89e5fd744008240ecd668272a4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 24 } } ignore_live_leader: false dest_uuid: "61c27cf3d4c846748ae6752e0fa652b9" is_pre_election: true
I20260811 01:31:21.490806 8804 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8970a89e5fd744008240ecd668272a4a in term 5.
W20260811 01:31:21.491315 8995 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:31:21.491417 8994 leader_election.cc:302] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9, 8970a89e5fd744008240ecd668272a4a; no voters:
I20260811 01:31:21.491927 9116 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260811 01:31:21.492157 9116 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:31:21.492297 9116 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 5 FOLLOWER]: Advancing to term 6
W20260811 01:31:21.493307 8995 leader_election.cc:334] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:31:21.496551 9116 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:21.497241 9116 leader_election.cc:288] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [CANDIDATE]: Term 6 election: Requested vote from peers 61c27cf3d4c846748ae6752e0fa652b9 (127.5.107.254:33755), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
I20260811 01:31:21.497596 8804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8970a89e5fd744008240ecd668272a4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 24 } } ignore_live_leader: false dest_uuid: "61c27cf3d4c846748ae6752e0fa652b9"
I20260811 01:31:21.497834 8804 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 5 FOLLOWER]: Advancing to term 6
W20260811 01:31:21.498632 8995 leader_election.cc:334] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:31:21.499951 8804 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8970a89e5fd744008240ecd668272a4a in term 6.
I20260811 01:31:21.500319 8994 leader_election.cc:302] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9, 8970a89e5fd744008240ecd668272a4a; no voters: c6f417c29b464762944a721d477b2b13
I20260811 01:31:21.500619 9116 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 6 FOLLOWER]: Leader election won for term 6
I20260811 01:31:21.501037 9116 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [term 6 LEADER]: Becoming Leader. State: Replica: 8970a89e5fd744008240ecd668272a4a, State: Running, Role: LEADER
I20260811 01:31:21.501487 9116 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 5.24, Last appended by leader: 24, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:21.503643 9126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8970a89e5fd744008240ecd668272a4a. Latest consensus state: current_term: 6 leader_uuid: "8970a89e5fd744008240ecd668272a4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } } }
I20260811 01:31:21.504011 9126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a [sys.catalog]: This master's current role is: LEADER
I20260811 01:31:21.504781 9132 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:31:21.507114 9132 catalog_manager.cc:669] Loaded metadata for table default.table_f7f5b09418c242d092b1948b866994e5 [id=69f9a5a340894b4bb41b150af86e0d8a]
I20260811 01:31:21.507567 9132 catalog_manager.cc:669] Loaded metadata for table default.table_b6731279f2fb4ef2a0a845c55840c58b [id=89b85d47429d40c787b453da0787febc]
I20260811 01:31:21.507915 9132 catalog_manager.cc:669] Loaded metadata for table default.table_c93618b4070e4974a6a794087d73fa73 [id=a65d01fd84fa41a3b31bf35b60f5cbaf]
I20260811 01:31:21.509406 9132 tablet_loader.cc:96] loaded metadata for tablet 01ab509f88494b5f998fd1705c5d0f59 (table default.table_c93618b4070e4974a6a794087d73fa73 [id=a65d01fd84fa41a3b31bf35b60f5cbaf])
I20260811 01:31:21.509903 9132 tablet_loader.cc:96] loaded metadata for tablet 24df17b4b73c4838aa887fe2c1d4bcd0 (table default.table_b6731279f2fb4ef2a0a845c55840c58b [id=89b85d47429d40c787b453da0787febc])
I20260811 01:31:21.510530 9132 tablet_loader.cc:96] loaded metadata for tablet 96e04578bb164dd48866d5925b000eff (table default.table_f7f5b09418c242d092b1948b866994e5 [id=69f9a5a340894b4bb41b150af86e0d8a])
I20260811 01:31:21.511090 9132 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:31:21.511933 9132 catalog_manager.cc:1259] Loaded cluster ID: 2f0039b0f9ec4f479f59cf7d8b74b7a2
I20260811 01:31:21.512029 9132 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:31:21.513214 9132 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:31:21.514132 9132 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8970a89e5fd744008240ecd668272a4a: Loaded TSK: 0
I20260811 01:31:21.514930 9132 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 01:31:21.516153 9132 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260811 01:31:21.517329 9132 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260811 01:31:21.570760 9013 master_service.cc:946] ReplaceTablet: received request to replace tablet a6bc3c3d5d904830bfda3966e47305da from {username='slave'} at 127.0.0.1:42108
I20260811 01:31:21.570894 9012 master_service.cc:946] ReplaceTablet: received request to replace tablet a6bc3c3d5d904830bfda3966e47305da from {username='slave'} at 127.0.0.1:42108
I20260811 01:31:21.571256 9014 master_service.cc:946] ReplaceTablet: received request to replace tablet 1cb26d431b24475a8cf62325267b0a3d from {username='slave'} at 127.0.0.1:42108
I20260811 01:31:21.571427 9011 master_service.cc:946] ReplaceTablet: received request to replace tablet 1cb26d431b24475a8cf62325267b0a3d from {username='slave'} at 127.0.0.1:42108
I20260811 01:31:21.572022 9009 master_service.cc:946] ReplaceTablet: received request to replace tablet a6bc3c3d5d904830bfda3966e47305da from {username='slave'} at 127.0.0.1:42108
I20260811 01:31:21.573009 9013 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:42108:
table { table_name: "default.Table_c93618b4070e4974a6a794087d73fa73" } new_table_name: "default.Table_9be7f6a4237648ab8b421f62026d3ad5" modify_external_catalogs: true
I20260811 01:31:21.572928 9014 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42108:
name: "default.Table_16925b327c134e08987029291b57034c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:31:21.574133 9011 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:42108:
table { table_name: "default.Table_f7f5b09418c242d092b1948b866994e5" } new_table_name: "default.Table_414082fd726f49d4a5926d9c9c739495" modify_external_catalogs: true
I20260811 01:31:21.573956 9008 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42108:
name: "default.Table_071a47d589c9407aa47c9d668c44cc23"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:31:21.575277 9009 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:42108:
table { table_name: "default.Table_b6731279f2fb4ef2a0a845c55840c58b" } new_table_name: "default.Table_ae0986f440b7487789e0721a121aa6d3" modify_external_catalogs: true
I20260811 01:31:21.574345 9002 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42108:
name: "default.Table_0bf1f8de8ab54285b6291b6c6e30b647"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:31:21.574018 9007 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42108:
name: "default.Table_e6b7b1e823f84e859b8956dc8c474b29"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:31:21.573390 9012 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42108:
name: "default.Table_9f9bb4e0b22447c79a5597bc7db4eec8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:31:21.575456 9006 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42108:
name: "default.Table_977e759aa4644f0091ecc8c4613cffc2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:31:21.575831 9003 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42108:
name: "default.Table_5e80cfaf385a44328ab0113196274005"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260811 01:31:21.580199 9107 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_d472cfe99d004f68bc6beda9b7c20bc4
*** Check failure stack trace: ***
@ 0x7f5d5b81d9ad google::LogMessage::Fail() at ??:0
@ 0x7f5d5b82088a google::LogMessage::SendToLog() at ??:0
@ 0x7f5d5b81d4de google::LogMessage::Flush() at ??:0
@ 0x7f5d5b81d6d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7f5d713bbba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7f5d713ba901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f5d713c166c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7f5d713c14e9 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f5d5e447eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f5d617216db start_thread at ??:0
@ 0x7f5d5a6b761f clone at ??:0
W20260811 01:31:21.791610 7131 connection.cc:535] client connection to 127.5.107.253:36427 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 01:31:21.792285 7131 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.5.107.253:36427)
W20260811 01:31:21.792994 7131 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.5.107.253:36427)
W20260811 01:31:21.793406 7131 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.5.107.253:36427)
I20260811 01:31:22.068262 7113 heartbeater.cc:503] Master 127.5.107.253:36427 was elected leader, sending a full tablet report...
I20260811 01:31:22.068372 6896 heartbeater.cc:503] Master 127.5.107.253:36427 was elected leader, sending a full tablet report...
I20260811 01:31:22.213546 8735 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:31:22.213716 8735 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:31:22.289397 9169 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:22.289650 9169 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:22.290570 9169 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:31:22.292939 8765 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:31:22.293164 8765 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:31:22.293301 8765 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:31:22.293568 9169 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 01:31:23.210238 5551 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZLXb2N/build/asan/bin/kudu pid 8735
I20260811 01:31:23.216944 8835 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:31:23.331179 5551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZLXb2N/build/asan/bin/kudu with pid 8735
I20260811 01:31:23.350666 9179 raft_consensus.cc:482] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:23.350907 9179 raft_consensus.cc:504] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:31:23.351617 9179 leader_election.cc:288] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:31:23.352547 6923 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
W20260811 01:31:23.352780 6920 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
I20260811 01:31:23.352946 6920 leader_election.cc:302] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:31:23.353276 9179 raft_consensus.cc:2726] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:31:24.214636 5551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZLXb2N/build/asan/bin/kudu
/tmp/dist-test-taskZLXb2N/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.107.254:33755
--webserver_interface=127.5.107.254
--webserver_port=45627
--builtin_ntp_servers=127.5.107.212:36421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519
--hive_metastore_uris=thrift://127.0.0.1:36237
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260811 01:31:24.554106 9186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:31:24.554399 9186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:31:24.554479 9186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:31:24.561720 9186 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:31:24.561869 9186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:31:24.561957 9186 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:31:24.562088 9186 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:31:24.571460 9186 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.107.212:36421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:36237
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.107.254:33755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.107.254
--webserver_port=45627
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:31:24.572968 9186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:31:24.574892 9186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:31:24.587102 9197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:31:24.587631 9198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:31:24.587945 9200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:31:24.588271 9186 server_base.cc:1034] running on GCE node
I20260811 01:31:24.589130 9186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:31:24.590680 9186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:31:24.591977 9186 hybrid_clock.cc:648] HybridClock initialized: now 1786411884591908 us; error 41 us; skew 500 ppm
I20260811 01:31:24.592357 9186 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:31:24.595165 9186 webserver.cc:460] Webserver started at http://127.5.107.254:45627/ using document root <none> and password file <none>
I20260811 01:31:24.595943 9186 fs_manager.cc:359] Metadata directory not provided
I20260811 01:31:24.596112 9186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:31:24.602283 9186 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 01:31:24.605276 9208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:31:24.606629 9186 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 01:31:24.606818 9186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/data,/tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/wal
uuid: "61c27cf3d4c846748ae6752e0fa652b9"
format_stamp: "Formatted at 2026-08-11 01:30:53 on dist-test-slave-2p8l"
I20260811 01:31:24.607766 9186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:31:24.623355 9186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:31:24.624186 9186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:31:24.624497 9186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:31:24.643118 9186 rpc_server.cc:307] RPC server started. Bound to: 127.5.107.254:33755
I20260811 01:31:24.643122 9300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.107.254:33755 every 8 connection(s)
I20260811 01:31:24.644783 9186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZLXb2N/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786411837546982-5551-0/minicluster-data/master-0/data/info.pb
I20260811 01:31:24.647359 5551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZLXb2N/build/asan/bin/kudu as pid 9186
I20260811 01:31:24.652534 9301 sys_catalog.cc:263] Verifying existing consensus state
I20260811 01:31:24.655978 9301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9: Bootstrap starting.
I20260811 01:31:24.664739 9309 raft_consensus.cc:482] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:24.664932 9309 raft_consensus.cc:504] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } }
I20260811 01:31:24.665436 9301 log.cc:824] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9: Log is configured to *not* fsync() on all Append() calls
I20260811 01:31:24.665861 9309 leader_election.cc:288] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:31:24.666965 6920 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
W20260811 01:31:24.667306 6923 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
I20260811 01:31:24.667452 6923 leader_election.cc:302] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:31:24.667752 9309 raft_consensus.cc:2726] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:31:24.685706 9301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=19 overwritten=0 applied=19 ignored=0} inserts{seen=12 ignored=0} mutations{seen=11 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 01:31:24.686340 9301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9: Bootstrap complete.
I20260811 01:31:24.693557 9301 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:24.694518 9301 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61c27cf3d4c846748ae6752e0fa652b9, State: Initialized, Role: FOLLOWER
I20260811 01:31:24.695127 9301 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 19, Last appended: 3.19, Last appended by leader: 19, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:24.695866 9311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 6 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } } }
I20260811 01:31:24.696208 9311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:31:24.696923 9301 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:31:24.708626 9337 catalog_manager.cc:1259] Loaded cluster ID: 2f0039b0f9ec4f479f59cf7d8b74b7a2
I20260811 01:31:24.708812 9337 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9: loading cluster ID for follower catalog manager: success
I20260811 01:31:24.713197 9337 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9: acquiring CA information for follower catalog manager: success
I20260811 01:31:24.716091 9337 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 01:31:24.723767 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:31:24.729861 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:31:24.787955 9344 raft_consensus.cc:482] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 992432fc73c94fcc9f2142b27ef725af)
I20260811 01:31:24.788193 9344 raft_consensus.cc:504] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:31:24.788951 9344 leader_election.cc:288] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:31:24.789955 6702 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
W20260811 01:31:24.790226 6699 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
I20260811 01:31:24.790390 6699 leader_election.cc:302] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:31:24.790668 9344 raft_consensus.cc:2726] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:31:24.823714 9344 raft_consensus.cc:482] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4a331de1488e4a0986b490917efea000)
I20260811 01:31:24.823920 9344 raft_consensus.cc:504] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:31:24.824636 9344 leader_election.cc:288] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:31:24.825465 6702 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
W20260811 01:31:24.825668 6699 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
I20260811 01:31:24.825814 6699 leader_election.cc:302] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:31:24.826114 9344 raft_consensus.cc:2726] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:31:24.921315 6891 heartbeater.cc:347] Connected to a master server at 127.5.107.254:33755
I20260811 01:31:24.925434 9241 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" instance_seqno: 1786411855798958) as {username='slave'} at 127.5.107.194:51141; Asking this server to re-register.
I20260811 01:31:24.926275 6891 heartbeater.cc:464] Registering TS with master...
I20260811 01:31:24.928184 9241 ts_manager.cc:194] Registered new tserver with Master: 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971)
I20260811 01:31:25.069365 7111 heartbeater.cc:347] Connected to a master server at 127.5.107.254:33755
I20260811 01:31:25.070808 9241 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" instance_seqno: 1786411856431448) as {username='slave'} at 127.5.107.195:59357; Asking this server to re-register.
I20260811 01:31:25.071395 7111 heartbeater.cc:464] Registering TS with master...
I20260811 01:31:25.072736 9241 ts_manager.cc:194] Registered new tserver with Master: 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
I20260811 01:31:25.105835 9311 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:25.106192 9311 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:25.107308 6657 heartbeater.cc:347] Connected to a master server at 127.5.107.254:33755
I20260811 01:31:25.108243 9311 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
I20260811 01:31:25.108462 9241 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "4a331de1488e4a0986b490917efea000" instance_seqno: 1786411855240795) as {username='slave'} at 127.5.107.193:53877; Asking this server to re-register.
W20260811 01:31:25.109017 9214 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
I20260811 01:31:25.109120 6657 heartbeater.cc:464] Registering TS with master...
I20260811 01:31:25.110458 9241 ts_manager.cc:194] Registered new tserver with Master: 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:31:25.110837 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:31:25.111109 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:31:25.111196 9214 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:31:25.111610 9311 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 01:31:25.623772 9347 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:25.623991 9347 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:25.624725 9347 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:31:25.631266 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:31:25.631512 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:31:25.631654 9214 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:31:25.631932 9347 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 01:31:26.352059 9355 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:26.352281 9355 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:26.352975 9355 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:31:26.355189 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:31:26.355367 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:31:26.355446 9214 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:31:26.355697 9355 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 01:31:27.080618 5551 master-stress-test.cc:578] Killing test thread 1/25
I20260811 01:31:27.700003 9359 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:27.700286 9359 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:27.700942 9359 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:31:27.704576 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:31:27.704797 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:31:27.704926 9214 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:31:27.705179 9359 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 01:31:28.185189 9364 raft_consensus.cc:482] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:28.185503 9364 raft_consensus.cc:504] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:31:28.186442 9364 leader_election.cc:288] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:31:28.187604 6923 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
W20260811 01:31:28.187798 6920 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
I20260811 01:31:28.187983 6920 leader_election.cc:302] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:31:28.188271 9364 raft_consensus.cc:2726] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:31:29.732966 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:31:29.733141 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:31:30.142740 9369 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:30.142984 9369 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:30.143841 9369 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:31:30.145180 9214 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111) [suppressed 7 similar messages]
W20260811 01:31:30.146904 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:31:30.147132 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:31:30.147264 9214 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:31:30.147527 9369 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 01:31:32.487239 9375 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:32.487479 9375 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:32.488193 9375 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:31:32.490425 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:31:32.490602 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:31:32.490684 9214 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:31:32.490916 9375 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 01:31:34.355290 9380 raft_consensus.cc:482] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4a331de1488e4a0986b490917efea000)
I20260811 01:31:34.355473 9380 raft_consensus.cc:504] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:31:34.356254 9380 leader_election.cc:288] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:31:34.357384 6699 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
W20260811 01:31:34.357604 6702 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
I20260811 01:31:34.357720 6702 leader_election.cc:302] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:31:34.357959 9380 raft_consensus.cc:2726] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:31:34.395552 9380 raft_consensus.cc:482] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 992432fc73c94fcc9f2142b27ef725af)
I20260811 01:31:34.395753 9380 raft_consensus.cc:504] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:31:34.396385 9380 leader_election.cc:288] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:31:34.397353 6699 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
W20260811 01:31:34.397512 6702 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
I20260811 01:31:34.397675 6702 leader_election.cc:302] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:31:34.397943 9380 raft_consensus.cc:2726] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:31:34.670475 9385 raft_consensus.cc:482] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:34.670730 9385 raft_consensus.cc:504] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } }
I20260811 01:31:34.671849 9385 leader_election.cc:288] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:31:34.672881 6920 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
W20260811 01:31:34.673070 6923 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
I20260811 01:31:34.673218 6923 leader_election.cc:302] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:31:34.673492 9385 raft_consensus.cc:2726] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:31:34.736259 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:31:34.736446 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:31:35.806272 9388 raft_consensus.cc:482] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:35.806546 9388 raft_consensus.cc:504] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:31:35.807204 9388 leader_election.cc:288] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:31:35.808247 6923 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
W20260811 01:31:35.808494 6920 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
I20260811 01:31:35.808641 6920 leader_election.cc:302] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:31:35.808923 9388 raft_consensus.cc:2726] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:31:36.701050 9390 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:36.701268 9390 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:36.701855 9390 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:31:36.703150 9214 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260811 01:31:36.704593 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:31:36.704818 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:31:36.704908 9214 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:31:36.705210 9390 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 01:31:39.729362 9394 raft_consensus.cc:482] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:39.729598 9394 raft_consensus.cc:504] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } }
I20260811 01:31:39.730330 9394 leader_election.cc:288] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:31:39.731344 6923 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
W20260811 01:31:39.731532 6920 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
I20260811 01:31:39.731712 6920 leader_election.cc:302] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:31:39.732012 9394 raft_consensus.cc:2726] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:31:39.739189 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:31:39.739367 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:31:41.122605 9396 raft_consensus.cc:482] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 992432fc73c94fcc9f2142b27ef725af)
I20260811 01:31:41.122859 9396 raft_consensus.cc:504] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:31:41.123557 9396 leader_election.cc:288] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:31:41.124509 6699 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
W20260811 01:31:41.124723 6702 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
I20260811 01:31:41.124871 6702 leader_election.cc:302] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:31:41.125150 9396 raft_consensus.cc:2726] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:31:41.196136 9397 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:41.196527 9397 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:41.197291 9397 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:31:41.200598 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:31:41.200855 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:31:41.200975 9214 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:31:41.201247 9397 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 01:31:42.658595 9405 raft_consensus.cc:482] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4a331de1488e4a0986b490917efea000)
I20260811 01:31:42.658768 9405 raft_consensus.cc:504] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:31:42.659376 9405 leader_election.cc:288] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:31:42.660267 6699 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
W20260811 01:31:42.660456 6702 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
I20260811 01:31:42.660606 6702 leader_election.cc:302] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:31:42.660872 9405 raft_consensus.cc:2726] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:31:44.742566 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:31:44.742748 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:31:48.690352 9416 raft_consensus.cc:482] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4a331de1488e4a0986b490917efea000)
I20260811 01:31:48.690613 9416 raft_consensus.cc:504] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:31:48.691372 9416 leader_election.cc:288] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:31:48.692332 6702 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
W20260811 01:31:48.692520 6699 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
I20260811 01:31:48.692667 6699 leader_election.cc:302] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:31:48.692929 9416 raft_consensus.cc:2726] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:31:49.745738 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:31:49.745906 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:31:53.345319 9421 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:53.345530 9421 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:53.346210 9421 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:31:53.347707 9214 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260811 01:31:53.351430 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:31:53.351660 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:31:53.351797 9214 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:31:53.352093 9421 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 01:31:53.557940 9428 raft_consensus.cc:482] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:53.558207 9428 raft_consensus.cc:504] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:31:53.559116 9428 leader_election.cc:288] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:31:53.560114 6920 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
W20260811 01:31:53.560289 6923 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
I20260811 01:31:53.560415 6923 leader_election.cc:302] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:31:53.560724 9428 raft_consensus.cc:2726] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:31:54.748657 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:31:54.748822 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:31:55.794973 9430 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:55.795208 9430 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:31:55.795925 9430 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:31:55.798521 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:31:55.798727 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:31:55.798818 9214 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:31:55.799090 9430 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 01:31:56.180339 9438 raft_consensus.cc:482] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 992432fc73c94fcc9f2142b27ef725af)
I20260811 01:31:56.180605 9438 raft_consensus.cc:504] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:31:56.181577 9438 leader_election.cc:288] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:31:56.182580 6702 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
W20260811 01:31:56.182749 6699 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
I20260811 01:31:56.182931 6699 leader_election.cc:302] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:31:56.183192 9438 raft_consensus.cc:2726] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260811 01:31:56.877403 6658 heartbeater.cc:644] Failed to heartbeat to 127.5.107.253:36427 (70 consecutive failures): Network error: Failed to ping master at 127.5.107.253:36427: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:31:57.163240 7113 heartbeater.cc:644] Failed to heartbeat to 127.5.107.253:36427 (70 consecutive failures): Network error: Failed to ping master at 127.5.107.253:36427: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:31:57.163244 6896 heartbeater.cc:644] Failed to heartbeat to 127.5.107.253:36427 (70 consecutive failures): Network error: Failed to ping master at 127.5.107.253:36427: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
I20260811 01:31:58.362598 9440 raft_consensus.cc:482] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:31:58.362859 9440 raft_consensus.cc:504] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } }
I20260811 01:31:58.363602 9440 leader_election.cc:288] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:31:58.364665 6920 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
W20260811 01:31:58.364845 6923 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
I20260811 01:31:58.364950 6923 leader_election.cc:302] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:31:58.365271 9440 raft_consensus.cc:2726] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:31:59.204089 9443 raft_consensus.cc:482] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4a331de1488e4a0986b490917efea000)
I20260811 01:31:59.204416 9443 raft_consensus.cc:504] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:31:59.205322 9443 leader_election.cc:288] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:31:59.206271 6702 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
W20260811 01:31:59.206470 6699 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
I20260811 01:31:59.206634 6699 leader_election.cc:302] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:31:59.206899 9443 raft_consensus.cc:2726] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:31:59.751487 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:31:59.751652 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:32:00.839448 9446 raft_consensus.cc:482] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:32:00.839655 9446 raft_consensus.cc:504] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:32:00.840193 9446 leader_election.cc:288] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:32:00.841245 6920 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
W20260811 01:32:00.841487 6923 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
I20260811 01:32:00.841636 6923 leader_election.cc:302] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:32:00.841938 9446 raft_consensus.cc:2726] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:32:03.909363 9447 raft_consensus.cc:482] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:32:03.909569 9447 raft_consensus.cc:504] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:32:03.910305 9447 leader_election.cc:288] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:32:03.911362 6923 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
W20260811 01:32:03.911574 6920 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
I20260811 01:32:03.911736 6920 leader_election.cc:302] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:32:03.912009 9447 raft_consensus.cc:2726] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:32:04.754258 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:32:04.754451 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:32:09.756891 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:32:09.757090 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:32:11.541570 9456 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:32:11.541764 9456 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:32:11.542733 9456 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:32:11.543838 9214 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260811 01:32:11.545200 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:32:11.545370 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:32:11.545440 9214 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:32:11.545670 9456 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 01:32:13.751816 9466 raft_consensus.cc:482] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:32:13.752090 9466 raft_consensus.cc:504] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } }
I20260811 01:32:13.752980 9466 leader_election.cc:288] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:32:13.753970 6923 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
W20260811 01:32:13.754197 6920 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
I20260811 01:32:13.754341 6920 leader_election.cc:302] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:32:13.754629 9466 raft_consensus.cc:2726] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:32:14.760169 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:32:14.760329 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:32:14.904763 9468 raft_consensus.cc:482] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4a331de1488e4a0986b490917efea000)
I20260811 01:32:14.904964 9468 raft_consensus.cc:504] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:32:14.905833 9468 leader_election.cc:288] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:32:14.906910 6699 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
W20260811 01:32:14.907086 6702 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
I20260811 01:32:14.907222 6702 leader_election.cc:302] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:32:14.907454 9468 raft_consensus.cc:2726] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:32:15.192731 9468 raft_consensus.cc:482] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 992432fc73c94fcc9f2142b27ef725af)
I20260811 01:32:15.192948 9468 raft_consensus.cc:504] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:32:15.193607 9468 leader_election.cc:288] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:32:15.194607 6699 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
W20260811 01:32:15.194826 6702 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
I20260811 01:32:15.194985 6702 leader_election.cc:302] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:32:15.195273 9468 raft_consensus.cc:2726] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:32:16.767100 9469 raft_consensus.cc:482] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:32:16.767299 9469 raft_consensus.cc:504] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:32:16.767894 9469 leader_election.cc:288] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:32:16.768980 6920 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
W20260811 01:32:16.769181 6923 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
I20260811 01:32:16.769328 6923 leader_election.cc:302] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:32:16.769619 9469 raft_consensus.cc:2726] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:32:19.763139 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:32:19.763335 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:32:20.209021 9475 raft_consensus.cc:482] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 992432fc73c94fcc9f2142b27ef725af)
I20260811 01:32:20.209322 9475 raft_consensus.cc:504] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:32:20.210103 9475 leader_election.cc:288] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:32:20.211172 6699 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
W20260811 01:32:20.211380 6702 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
I20260811 01:32:20.211532 6702 leader_election.cc:302] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:32:20.211799 9475 raft_consensus.cc:2726] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:32:20.388401 9476 raft_consensus.cc:482] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:32:20.388616 9476 raft_consensus.cc:504] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:32:20.389354 9476 leader_election.cc:288] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:32:20.390390 6923 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
W20260811 01:32:20.390655 6920 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
I20260811 01:32:20.390827 6920 leader_election.cc:302] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:32:20.391160 9476 raft_consensus.cc:2726] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:32:22.744513 9478 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:32:22.744894 9478 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:32:22.745869 9478 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:32:22.747195 9214 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 01:32:22.748883 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:32:22.749190 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:32:22.749326 9214 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:32:22.749615 9478 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 01:32:24.766247 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:32:24.766403 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:32:29.205307 9485 raft_consensus.cc:482] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:32:29.205593 9485 raft_consensus.cc:504] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } }
I20260811 01:32:29.206575 9485 leader_election.cc:288] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:32:29.207574 6923 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
W20260811 01:32:29.207798 6920 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
I20260811 01:32:29.207926 6920 leader_election.cc:302] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:32:29.208161 9485 raft_consensus.cc:2726] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:32:29.769204 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:32:29.769369 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:32:30.882922 9487 raft_consensus.cc:482] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 992432fc73c94fcc9f2142b27ef725af)
I20260811 01:32:30.883234 9487 raft_consensus.cc:504] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:32:30.884207 9487 leader_election.cc:288] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:32:30.885353 6702 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
W20260811 01:32:30.885635 6699 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
I20260811 01:32:30.885812 6699 leader_election.cc:302] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:32:30.886175 9487 raft_consensus.cc:2726] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:32:31.180380 9488 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:32:31.180615 9488 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:32:31.181336 9488 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:32:31.182876 9214 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 01:32:31.185243 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:32:31.185493 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:32:31.185607 9214 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:32:31.185871 9488 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 01:32:32.467130 9495 raft_consensus.cc:482] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:32:32.467347 9495 raft_consensus.cc:504] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } }
I20260811 01:32:32.468093 9495 leader_election.cc:288] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:32:32.469075 6923 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
W20260811 01:32:32.469301 6920 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
I20260811 01:32:32.469408 6920 leader_election.cc:302] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:32:32.469636 9495 raft_consensus.cc:2726] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:32:33.084641 9499 raft_consensus.cc:482] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4a331de1488e4a0986b490917efea000)
I20260811 01:32:33.084944 9499 raft_consensus.cc:504] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:32:33.085635 9499 leader_election.cc:288] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:32:33.086642 6702 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
W20260811 01:32:33.086855 6699 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
I20260811 01:32:33.086982 6699 leader_election.cc:302] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:32:33.087266 9499 raft_consensus.cc:2726] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:32:34.775238 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:32:34.775441 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:32:36.705762 9502 raft_consensus.cc:482] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:32:36.706014 9502 raft_consensus.cc:504] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:32:36.706760 9502 leader_election.cc:288] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:32:36.707870 6923 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
W20260811 01:32:36.708096 6920 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
I20260811 01:32:36.708212 6920 leader_election.cc:302] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:32:36.708469 9502 raft_consensus.cc:2726] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:32:39.778162 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:32:39.778355 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:32:39.928046 9504 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:32:39.928303 9504 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:32:39.929059 9504 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:32:39.931031 9214 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 01:32:39.932338 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:32:39.932555 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:32:39.932639 9214 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:32:39.932863 9504 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 01:32:42.987104 9510 raft_consensus.cc:482] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:32:42.987349 9510 raft_consensus.cc:504] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:32:42.988003 9510 leader_election.cc:288] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:32:42.989161 6920 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
W20260811 01:32:42.989414 6923 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
I20260811 01:32:42.989579 6923 leader_election.cc:302] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:32:42.989873 9510 raft_consensus.cc:2726] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:32:44.211632 9511 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:32:44.211875 9511 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:32:44.212626 9511 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:32:44.215274 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:32:44.215441 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:32:44.215549 9214 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:32:44.215786 9511 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 01:32:44.780822 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:32:44.781029 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:32:47.883141 9517 raft_consensus.cc:482] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 992432fc73c94fcc9f2142b27ef725af)
I20260811 01:32:47.883396 9517 raft_consensus.cc:504] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:32:47.884586 9517 leader_election.cc:288] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:32:47.885694 6699 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
W20260811 01:32:47.885840 6702 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
I20260811 01:32:47.886018 6702 leader_election.cc:302] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:32:47.886296 9517 raft_consensus.cc:2726] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:32:48.534180 9518 raft_consensus.cc:482] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:32:48.534446 9518 raft_consensus.cc:504] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } }
I20260811 01:32:48.535352 9518 leader_election.cc:288] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:32:48.536260 6923 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
W20260811 01:32:48.536409 6920 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
I20260811 01:32:48.536592 6920 leader_election.cc:302] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:32:48.536867 9518 raft_consensus.cc:2726] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:32:49.783993 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:32:49.784231 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:32:49.952479 9520 raft_consensus.cc:482] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4a331de1488e4a0986b490917efea000)
I20260811 01:32:49.952745 9520 raft_consensus.cc:504] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:32:49.953462 9520 leader_election.cc:288] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:32:49.954555 6699 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
W20260811 01:32:49.954754 6702 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
I20260811 01:32:49.954888 6702 leader_election.cc:302] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:32:49.955181 9520 raft_consensus.cc:2726] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:32:54.787422 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:32:54.787637 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:32:56.399509 9523 raft_consensus.cc:482] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:32:56.399758 9523 raft_consensus.cc:504] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:32:56.400522 9523 leader_election.cc:288] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:32:56.401525 6923 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
W20260811 01:32:56.401726 6920 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
I20260811 01:32:56.401886 6920 leader_election.cc:302] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:32:56.402235 9523 raft_consensus.cc:2726] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 01:32:56.975914 6654 heartbeater.cc:644] Failed to heartbeat to 127.5.107.252:45519 (204 consecutive failures): Network error: Failed to ping master at 127.5.107.252:45519: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
W20260811 01:32:57.317246 6896 heartbeater.cc:644] Failed to heartbeat to 127.5.107.253:36427 (190 consecutive failures): Network error: Failed to ping master at 127.5.107.253:36427: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:32:57.317797 7113 heartbeater.cc:644] Failed to heartbeat to 127.5.107.253:36427 (190 consecutive failures): Network error: Failed to ping master at 127.5.107.253:36427: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
I20260811 01:32:59.790668 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:32:59.790832 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:33:00.828325 9526 raft_consensus.cc:482] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 992432fc73c94fcc9f2142b27ef725af)
I20260811 01:33:00.828780 9526 raft_consensus.cc:504] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:33:00.829725 9526 leader_election.cc:288] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:33:00.830739 6702 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
W20260811 01:33:00.830963 6699 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
I20260811 01:33:00.831175 6699 leader_election.cc:302] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:33:00.831461 9526 raft_consensus.cc:2726] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:33:02.783382 9527 raft_consensus.cc:482] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:33:02.783641 9527 raft_consensus.cc:504] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } }
I20260811 01:33:02.784543 9527 leader_election.cc:288] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:33:02.785602 6920 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
W20260811 01:33:02.786149 6923 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
I20260811 01:33:02.786284 6923 leader_election.cc:302] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:33:02.786566 9527 raft_consensus.cc:2726] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:33:03.661810 9528 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:33:03.662132 9528 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:33:03.663022 9528 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:33:03.664252 9214 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260811 01:33:03.665789 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:33:03.666121 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:33:03.666262 9214 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:33:03.666560 9528 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 01:33:04.793797 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:33:04.793964 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:33:06.612365 9539 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:33:06.612627 9539 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:33:06.613356 9539 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:33:06.616526 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:33:06.616714 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:33:06.616786 9214 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:33:06.617010 9539 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 01:33:06.799928 9545 raft_consensus.cc:482] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4a331de1488e4a0986b490917efea000)
I20260811 01:33:06.800213 9545 raft_consensus.cc:504] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:33:06.801435 9545 leader_election.cc:288] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:33:06.802444 6702 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
W20260811 01:33:06.802623 6699 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
I20260811 01:33:06.802809 6699 leader_election.cc:302] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:33:06.803092 9545 raft_consensus.cc:2726] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:33:09.797102 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:33:09.797317 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:33:10.935189 9549 raft_consensus.cc:482] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 992432fc73c94fcc9f2142b27ef725af)
I20260811 01:33:10.935460 9549 raft_consensus.cc:504] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:33:10.936173 9549 leader_election.cc:288] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:33:10.937237 6702 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
W20260811 01:33:10.937398 6699 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
I20260811 01:33:10.937744 6699 leader_election.cc:302] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:33:10.938084 9549 raft_consensus.cc:2726] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:33:11.914014 9553 raft_consensus.cc:482] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:33:11.914292 9553 raft_consensus.cc:504] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:33:11.915083 9553 leader_election.cc:288] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:33:11.916230 6923 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
W20260811 01:33:11.916407 6920 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
I20260811 01:33:11.916497 6920 leader_election.cc:302] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:33:11.916743 9553 raft_consensus.cc:2726] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:33:14.800287 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:33:14.800530 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:33:15.444660 9556 raft_consensus.cc:482] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4a331de1488e4a0986b490917efea000)
I20260811 01:33:15.444877 9556 raft_consensus.cc:504] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:33:15.445634 9556 leader_election.cc:288] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:33:15.446559 6702 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
W20260811 01:33:15.446749 6699 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
I20260811 01:33:15.446916 6699 leader_election.cc:302] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:33:15.447189 9556 raft_consensus.cc:2726] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:33:15.600572 9557 raft_consensus.cc:482] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:33:15.600812 9557 raft_consensus.cc:504] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } }
I20260811 01:33:15.601467 9557 leader_election.cc:288] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:33:15.602308 6923 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
W20260811 01:33:15.602531 6920 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
I20260811 01:33:15.602671 6920 leader_election.cc:302] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:33:15.602933 9557 raft_consensus.cc:2726] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:33:18.115612 9563 raft_consensus.cc:482] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:33:18.115895 9563 raft_consensus.cc:504] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:33:18.116672 9563 leader_election.cc:288] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:33:18.117673 6923 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
W20260811 01:33:18.117846 6920 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
I20260811 01:33:18.117949 6920 leader_election.cc:302] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:33:18.118225 9563 raft_consensus.cc:2726] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:33:19.803651 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:33:19.803857 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:33:21.244549 9570 raft_consensus.cc:482] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 992432fc73c94fcc9f2142b27ef725af)
I20260811 01:33:21.244812 9570 raft_consensus.cc:504] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:33:21.245752 9570 leader_election.cc:288] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:33:21.246827 6699 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
W20260811 01:33:21.246996 6702 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
I20260811 01:33:21.247186 6702 leader_election.cc:302] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:33:21.247463 9570 raft_consensus.cc:2726] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:33:24.455484 9572 raft_consensus.cc:482] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4a331de1488e4a0986b490917efea000)
I20260811 01:33:24.455681 9572 raft_consensus.cc:504] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:33:24.456503 9572 leader_election.cc:288] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:33:24.457603 6699 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
W20260811 01:33:24.457788 6702 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
I20260811 01:33:24.457913 6702 leader_election.cc:302] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:33:24.458158 9572 raft_consensus.cc:2726] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:33:24.802838 9218 maintenance_manager.cc:419] P 61c27cf3d4c846748ae6752e0fa652b9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033363
I20260811 01:33:24.809273 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:33:24.809474 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:33:24.845139 9217 maintenance_manager.cc:643] P 61c27cf3d4c846748ae6752e0fa652b9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.041s user 0.026s sys 0.000s Metrics: {"bytes_written":8807,"cfile_init":1,"dirs.queue_time_us":393,"dirs.run_cpu_time_us":196,"dirs.run_wall_time_us":1033,"drs_written":1,"lbm_read_time_us":104,"lbm_reads_lt_1ms":4,"lbm_write_time_us":659,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":12,"thread_start_us":472,"threads_started":2,"wal-append.queue_time_us":337}
I20260811 01:33:24.846408 9218 maintenance_manager.cc:419] P 61c27cf3d4c846748ae6752e0fa652b9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 828 bytes on disk
I20260811 01:33:24.847147 9217 maintenance_manager.cc:643] P 61c27cf3d4c846748ae6752e0fa652b9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":81,"lbm_reads_lt_1ms":4}
I20260811 01:33:25.257956 9579 raft_consensus.cc:482] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 992432fc73c94fcc9f2142b27ef725af)
I20260811 01:33:25.258227 9579 raft_consensus.cc:504] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:33:25.258934 9579 leader_election.cc:288] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:33:25.260002 6699 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
W20260811 01:33:25.260185 6702 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
I20260811 01:33:25.260310 6702 leader_election.cc:302] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:33:25.260571 9579 raft_consensus.cc:2726] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:33:25.716349 9582 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:33:25.716602 9582 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:33:25.717437 9582 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:33:25.718743 9214 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260811 01:33:25.723356 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:33:25.723644 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:33:25.723753 9214 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:33:25.724032 9582 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 01:33:29.812578 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:33:29.812804 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:33:31.064862 9603 raft_consensus.cc:482] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:33:31.065230 9603 raft_consensus.cc:504] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } }
I20260811 01:33:31.066619 9603 leader_election.cc:288] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:33:31.067413 6923 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
W20260811 01:33:31.067642 6920 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
I20260811 01:33:31.067787 6920 leader_election.cc:302] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:33:31.068058 9603 raft_consensus.cc:2726] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:33:33.411348 9620 raft_consensus.cc:482] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 992432fc73c94fcc9f2142b27ef725af)
I20260811 01:33:33.411571 9620 raft_consensus.cc:504] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:33:33.412398 9620 leader_election.cc:288] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:33:33.413597 6699 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
W20260811 01:33:33.413812 6702 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
I20260811 01:33:33.413976 6702 leader_election.cc:302] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:33:33.414237 9620 raft_consensus.cc:2726] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:33:33.632658 9621 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:33:33.632936 9621 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:33:33.633715 9621 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:33:33.635067 9214 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 01:33:33.637113 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:33:33.637344 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:33:33.637431 9214 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:33:33.637748 9621 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 01:33:34.815815 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:33:34.816000 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:33:35.939605 9648 raft_consensus.cc:482] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:33:35.939828 9648 raft_consensus.cc:504] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:33:35.940557 9648 leader_election.cc:288] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:33:35.941665 6920 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
W20260811 01:33:35.941895 6923 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
I20260811 01:33:35.941998 6923 leader_election.cc:302] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:33:35.942309 9648 raft_consensus.cc:2726] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:33:39.819192 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:33:39.819413 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:33:42.940891 9694 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:33:42.941228 9694 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:33:42.942046 9694 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:33:42.943305 9214 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 01:33:42.945349 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:33:42.945590 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:33:42.945691 9214 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:33:42.946030 9694 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 01:33:44.003952 9705 raft_consensus.cc:482] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4a331de1488e4a0986b490917efea000)
I20260811 01:33:44.004236 9705 raft_consensus.cc:504] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:33:44.005179 9705 leader_election.cc:288] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:33:44.006356 6702 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
W20260811 01:33:44.006560 6699 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
I20260811 01:33:44.006718 6699 leader_election.cc:302] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:33:44.006971 9705 raft_consensus.cc:2726] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:33:44.822500 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:33:44.822669 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:33:48.031200 9736 raft_consensus.cc:482] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:33:48.031448 9736 raft_consensus.cc:504] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } }
I20260811 01:33:48.032452 9736 leader_election.cc:288] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:33:48.033321 6920 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
W20260811 01:33:48.033705 6923 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
I20260811 01:33:48.033839 6923 leader_election.cc:302] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:33:48.034175 9736 raft_consensus.cc:2726] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:33:48.427981 9736 raft_consensus.cc:482] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:33:48.428246 9736 raft_consensus.cc:504] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:33:48.428979 9736 leader_election.cc:288] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:33:48.429978 6923 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
W20260811 01:33:48.430198 6920 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
I20260811 01:33:48.430316 6920 leader_election.cc:302] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:33:48.430531 9736 raft_consensus.cc:2726] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:33:49.826218 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:33:49.826455 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:33:51.853609 9768 raft_consensus.cc:482] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 992432fc73c94fcc9f2142b27ef725af)
I20260811 01:33:51.853917 9768 raft_consensus.cc:504] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:33:51.854787 9768 leader_election.cc:288] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:33:51.855899 6702 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
W20260811 01:33:51.856099 6699 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
I20260811 01:33:51.856268 6699 leader_election.cc:302] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:33:51.856529 9768 raft_consensus.cc:2726] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:33:54.829584 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:33:54.829752 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 01:33:57.138674 6654 heartbeater.cc:644] Failed to heartbeat to 127.5.107.252:45519 (324 consecutive failures): Network error: Failed to ping master at 127.5.107.252:45519: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
W20260811 01:33:57.485617 7113 heartbeater.cc:644] Failed to heartbeat to 127.5.107.253:36427 (310 consecutive failures): Network error: Failed to ping master at 127.5.107.253:36427: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:33:57.485625 6896 heartbeater.cc:644] Failed to heartbeat to 127.5.107.253:36427 (310 consecutive failures): Network error: Failed to ping master at 127.5.107.253:36427: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
I20260811 01:33:58.098875 9809 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:33:58.099184 9809 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:33:58.100270 9809 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:33:58.101771 9214 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 01:33:58.103215 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:33:58.103448 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:33:58.103545 9214 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:33:58.103826 9809 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 01:33:58.726651 9822 raft_consensus.cc:482] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4a331de1488e4a0986b490917efea000)
I20260811 01:33:58.726945 9822 raft_consensus.cc:504] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:33:58.727586 9822 leader_election.cc:288] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:33:58.728643 6699 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
W20260811 01:33:58.728824 6702 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
I20260811 01:33:58.728964 6702 leader_election.cc:302] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:33:58.729218 9822 raft_consensus.cc:2726] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:33:59.832847 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:33:59.833061 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:34:03.038362 9850 raft_consensus.cc:482] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4a331de1488e4a0986b490917efea000)
I20260811 01:34:03.038656 9850 raft_consensus.cc:504] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:34:03.039587 9850 leader_election.cc:288] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:34:03.040706 6699 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
W20260811 01:34:03.040939 6702 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
I20260811 01:34:03.041078 6702 leader_election.cc:302] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:34:03.041337 9850 raft_consensus.cc:2726] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:34:04.836937 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:34:04.837139 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:34:05.283648 9861 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:34:05.283890 9861 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:34:05.284705 9861 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:34:05.285640 9214 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 01:34:05.287170 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:34:05.287403 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:34:05.287521 9214 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:34:05.287813 9861 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 01:34:07.007009 9879 raft_consensus.cc:482] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 992432fc73c94fcc9f2142b27ef725af)
I20260811 01:34:07.007197 9879 raft_consensus.cc:504] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:34:07.007930 9879 leader_election.cc:288] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:34:07.008966 6699 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
W20260811 01:34:07.009168 6702 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
I20260811 01:34:07.009298 6702 leader_election.cc:302] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:34:07.009541 9879 raft_consensus.cc:2726] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:34:07.615150 9883 raft_consensus.cc:482] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:34:07.615401 9883 raft_consensus.cc:504] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } }
I20260811 01:34:07.616393 9883 leader_election.cc:288] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:34:07.617489 6923 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
W20260811 01:34:07.617748 6920 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
I20260811 01:34:07.617856 6920 leader_election.cc:302] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:34:07.618196 9883 raft_consensus.cc:2726] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:34:08.358312 9888 raft_consensus.cc:482] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:34:08.358495 9888 raft_consensus.cc:504] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:34:08.359165 9888 leader_election.cc:288] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:34:08.360196 6920 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
W20260811 01:34:08.360380 6923 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
I20260811 01:34:08.360483 6923 leader_election.cc:302] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:34:08.360709 9888 raft_consensus.cc:2726] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:34:08.751097 9891 raft_consensus.cc:482] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4a331de1488e4a0986b490917efea000)
I20260811 01:34:08.751315 9891 raft_consensus.cc:504] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:34:08.751906 9891 leader_election.cc:288] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:34:08.752897 6702 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
W20260811 01:34:08.753106 6699 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
I20260811 01:34:08.753202 6699 leader_election.cc:302] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:34:08.753506 9891 raft_consensus.cc:2726] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:34:09.840405 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:34:09.840629 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:34:11.531716 9906 raft_consensus.cc:482] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 992432fc73c94fcc9f2142b27ef725af)
I20260811 01:34:11.531904 9906 raft_consensus.cc:504] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:34:11.532569 9906 leader_election.cc:288] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:34:11.533607 6702 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
W20260811 01:34:11.533805 6699 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
I20260811 01:34:11.533921 6699 leader_election.cc:302] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:34:11.534192 9906 raft_consensus.cc:2726] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:34:13.656364 9922 raft_consensus.cc:482] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:34:13.656680 9922 raft_consensus.cc:504] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } }
I20260811 01:34:13.657487 9922 leader_election.cc:288] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:34:13.658548 6923 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
W20260811 01:34:13.658756 6920 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
I20260811 01:34:13.658893 6920 leader_election.cc:302] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:34:13.659147 9922 raft_consensus.cc:2726] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:34:14.843645 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:34:14.843863 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:34:19.782896 9958 raft_consensus.cc:482] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:34:19.783146 9958 raft_consensus.cc:504] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:34:19.783902 9958 leader_election.cc:288] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:34:19.784917 6923 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
W20260811 01:34:19.785108 6920 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
I20260811 01:34:19.785239 6920 leader_election.cc:302] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:34:19.785501 9958 raft_consensus.cc:2726] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:34:19.847285 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:34:19.847500 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:34:20.022569 9958 raft_consensus.cc:482] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:34:20.022792 9958 raft_consensus.cc:504] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } }
I20260811 01:34:20.023474 9958 leader_election.cc:288] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:34:20.024575 6920 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
W20260811 01:34:20.024787 6923 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
I20260811 01:34:20.024890 6923 leader_election.cc:302] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:34:20.025130 9958 raft_consensus.cc:2726] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:34:23.493490 9985 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:34:23.493738 9985 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:34:23.494941 9985 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:34:23.497176 9214 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 01:34:23.498888 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:34:23.499053 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:34:23.499130 9214 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:34:23.499409 9985 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 01:34:24.850597 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:34:24.850816 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:34:27.256819 10018 raft_consensus.cc:482] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4a331de1488e4a0986b490917efea000)
I20260811 01:34:27.257128 10018 raft_consensus.cc:504] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:34:27.258076 10018 leader_election.cc:288] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:34:27.259050 6699 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
W20260811 01:34:27.259232 6702 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
I20260811 01:34:27.259349 6702 leader_election.cc:302] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:34:27.259589 10018 raft_consensus.cc:2726] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:34:28.767598 10027 raft_consensus.cc:482] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 992432fc73c94fcc9f2142b27ef725af)
I20260811 01:34:28.767791 10027 raft_consensus.cc:504] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:34:28.768386 10027 leader_election.cc:288] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:34:28.769428 6702 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
W20260811 01:34:28.769652 6699 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
I20260811 01:34:28.769822 6699 leader_election.cc:302] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:34:28.770113 10027 raft_consensus.cc:2726] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:34:29.853749 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:34:29.853925 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:34:32.059759 10048 raft_consensus.cc:482] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 992432fc73c94fcc9f2142b27ef725af)
I20260811 01:34:32.059993 10048 raft_consensus.cc:504] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:34:32.060662 10048 leader_election.cc:288] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:34:32.061775 6699 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
W20260811 01:34:32.062016 6702 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
I20260811 01:34:32.062170 6702 leader_election.cc:302] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:34:32.062438 10048 raft_consensus.cc:2726] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:34:34.236824 10060 raft_consensus.cc:482] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4a331de1488e4a0986b490917efea000)
I20260811 01:34:34.237035 10060 raft_consensus.cc:504] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:34:34.237668 10060 leader_election.cc:288] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:34:34.238742 6699 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
W20260811 01:34:34.239248 6702 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
I20260811 01:34:34.239394 6702 leader_election.cc:302] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:34:34.239675 10060 raft_consensus.cc:2726] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:34:34.857197 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:34:34.857383 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:34:35.986867 10076 raft_consensus.cc:482] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:34:35.987107 10076 raft_consensus.cc:504] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } }
I20260811 01:34:35.988054 10076 leader_election.cc:288] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:34:35.989012 6920 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
W20260811 01:34:35.989236 6923 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
I20260811 01:34:35.989374 6923 leader_election.cc:302] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:34:35.989663 10076 raft_consensus.cc:2726] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:34:37.365459 10080 raft_consensus.cc:482] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 992432fc73c94fcc9f2142b27ef725af)
I20260811 01:34:37.365756 10080 raft_consensus.cc:504] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:34:37.366425 10080 leader_election.cc:288] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:34:37.367538 6699 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
W20260811 01:34:37.367756 6702 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
I20260811 01:34:37.367897 6702 leader_election.cc:302] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:34:37.368196 10080 raft_consensus.cc:2726] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:34:38.821986 10096 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:34:38.822268 10096 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:34:38.823115 10096 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:34:38.827106 9214 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 01:34:38.828600 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:34:38.828821 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:34:38.828912 9214 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:34:38.829254 10096 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 01:34:38.906617 10102 raft_consensus.cc:482] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:34:38.906823 10102 raft_consensus.cc:504] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:34:38.907578 10102 leader_election.cc:288] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:34:38.908607 6923 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
W20260811 01:34:38.908810 6920 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
I20260811 01:34:38.908995 6920 leader_election.cc:302] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:34:38.909250 10102 raft_consensus.cc:2726] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:34:39.860224 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:34:39.860404 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:34:43.831907 10138 raft_consensus.cc:482] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4a331de1488e4a0986b490917efea000)
I20260811 01:34:43.832129 10138 raft_consensus.cc:504] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:34:43.833017 10138 leader_election.cc:288] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:34:43.833933 6702 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
W20260811 01:34:43.834132 6699 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
I20260811 01:34:43.834239 6699 leader_election.cc:302] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:34:43.834446 10138 raft_consensus.cc:2726] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:34:44.217103 10138 raft_consensus.cc:482] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 992432fc73c94fcc9f2142b27ef725af)
I20260811 01:34:44.217315 10138 raft_consensus.cc:504] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:34:44.217953 10138 leader_election.cc:288] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:34:44.218992 6702 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
W20260811 01:34:44.219201 6699 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
I20260811 01:34:44.219381 6699 leader_election.cc:302] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:34:44.219638 10138 raft_consensus.cc:2726] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:34:44.863515 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:34:44.863754 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:34:48.505196 10158 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:34:48.505404 10158 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:34:48.506026 10158 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:34:48.507385 9214 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 01:34:48.511252 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:34:48.511502 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:34:48.511639 9214 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:34:48.511911 10158 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 01:34:49.225149 10170 raft_consensus.cc:482] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:34:49.225425 10170 raft_consensus.cc:504] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } }
I20260811 01:34:49.226380 10170 leader_election.cc:288] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:34:49.227283 6920 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
W20260811 01:34:49.227464 6923 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
I20260811 01:34:49.227670 6923 leader_election.cc:302] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:34:49.227947 10170 raft_consensus.cc:2726] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:34:49.866678 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:34:49.866838 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:34:54.870038 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:34:54.870261 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:34:55.045789 10209 raft_consensus.cc:482] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:34:55.046030 10209 raft_consensus.cc:504] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:34:55.047045 10209 leader_election.cc:288] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:34:55.048197 6920 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
W20260811 01:34:55.048364 6923 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
I20260811 01:34:55.048545 6923 leader_election.cc:302] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:34:55.048866 10209 raft_consensus.cc:2726] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:34:55.766690 10215 raft_consensus.cc:482] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:34:55.766878 10215 raft_consensus.cc:504] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } }
I20260811 01:34:55.767504 10215 leader_election.cc:288] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:34:55.768486 6920 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
W20260811 01:34:55.768647 6923 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
I20260811 01:34:55.768770 6923 leader_election.cc:302] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:34:55.768999 10215 raft_consensus.cc:2726] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:34:56.553035 10219 raft_consensus.cc:482] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4a331de1488e4a0986b490917efea000)
I20260811 01:34:56.553390 10219 raft_consensus.cc:504] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:34:56.554248 10219 leader_election.cc:288] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:34:56.555301 6702 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
W20260811 01:34:56.555562 6699 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
I20260811 01:34:56.555696 6699 leader_election.cc:302] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:34:56.555912 10219 raft_consensus.cc:2726] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260811 01:34:57.301128 6654 heartbeater.cc:644] Failed to heartbeat to 127.5.107.252:45519 (444 consecutive failures): Network error: Failed to ping master at 127.5.107.252:45519: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
W20260811 01:34:57.648963 6896 heartbeater.cc:644] Failed to heartbeat to 127.5.107.253:36427 (430 consecutive failures): Network error: Failed to ping master at 127.5.107.253:36427: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:34:57.649013 7113 heartbeater.cc:644] Failed to heartbeat to 127.5.107.253:36427 (430 consecutive failures): Network error: Failed to ping master at 127.5.107.253:36427: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
I20260811 01:34:59.873206 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:34:59.873371 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:35:00.891769 10255 raft_consensus.cc:482] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:35:00.892086 10255 raft_consensus.cc:504] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } }
I20260811 01:35:00.892864 10255 leader_election.cc:288] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:35:00.893924 6920 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
W20260811 01:35:00.894120 6923 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
I20260811 01:35:00.894304 6923 leader_election.cc:302] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:35:00.894574 10255 raft_consensus.cc:2726] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:35:02.540395 10265 raft_consensus.cc:482] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 992432fc73c94fcc9f2142b27ef725af)
I20260811 01:35:02.540705 10265 raft_consensus.cc:504] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:35:02.541630 10265 leader_election.cc:288] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:35:02.542657 6699 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
W20260811 01:35:02.542857 6702 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
I20260811 01:35:02.542979 6702 leader_election.cc:302] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:35:02.543202 10265 raft_consensus.cc:2726] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:35:03.566696 10277 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:35:03.566948 10277 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:35:03.567847 10277 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:35:03.569626 9214 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 01:35:03.571329 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:35:03.571661 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:35:03.571774 9214 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:35:03.572094 10277 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 01:35:04.876590 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:35:04.876765 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:35:08.140924 10310 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:35:08.141208 10310 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:35:08.141883 10310 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:35:08.144881 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:35:08.145124 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:35:08.145257 9214 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:35:08.145562 10310 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 01:35:08.768201 10320 raft_consensus.cc:482] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4a331de1488e4a0986b490917efea000)
I20260811 01:35:08.768476 10320 raft_consensus.cc:504] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:35:08.769212 10320 leader_election.cc:288] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:35:08.770396 6702 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
W20260811 01:35:08.770566 6699 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
I20260811 01:35:08.770793 6699 leader_election.cc:302] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:35:08.771081 10320 raft_consensus.cc:2726] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:35:09.879433 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:35:09.879601 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:35:11.879215 10337 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:35:11.879446 10337 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:35:11.880088 10337 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:35:11.881356 9214 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260811 01:35:11.883312 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:35:11.883548 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:35:11.883680 9214 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:35:11.883975 10337 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 01:35:14.109838 10360 raft_consensus.cc:482] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:35:14.110193 10360 raft_consensus.cc:504] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:35:14.111353 10360 leader_election.cc:288] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:35:14.112465 6920 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
W20260811 01:35:14.112656 6923 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
I20260811 01:35:14.112845 6923 leader_election.cc:302] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:35:14.113116 10360 raft_consensus.cc:2726] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:35:14.882779 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:35:14.882948 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:35:15.345549 10366 raft_consensus.cc:482] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 992432fc73c94fcc9f2142b27ef725af)
I20260811 01:35:15.345836 10366 raft_consensus.cc:504] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:35:15.346866 10366 leader_election.cc:288] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:35:15.347903 6702 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
W20260811 01:35:15.348099 6699 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
I20260811 01:35:15.348282 6699 leader_election.cc:302] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:35:15.348547 10366 raft_consensus.cc:2726] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:35:16.137089 10372 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:35:16.137337 10372 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:35:16.138008 10372 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:35:16.140864 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:35:16.141124 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:35:16.141224 9214 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:35:16.141531 10372 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 01:35:19.565361 10391 raft_consensus.cc:482] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:35:19.565618 10391 raft_consensus.cc:504] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } }
I20260811 01:35:19.566326 10391 leader_election.cc:288] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:35:19.567327 6920 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
W20260811 01:35:19.567526 6923 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
I20260811 01:35:19.567647 6923 leader_election.cc:302] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:35:19.567890 10391 raft_consensus.cc:2726] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:35:19.885877 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:35:19.886111 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:35:22.439054 10408 raft_consensus.cc:482] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 992432fc73c94fcc9f2142b27ef725af)
I20260811 01:35:22.439250 10408 raft_consensus.cc:504] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:35:22.440083 10408 leader_election.cc:288] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:35:22.441011 6702 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
W20260811 01:35:22.441206 6699 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
I20260811 01:35:22.441455 6699 leader_election.cc:302] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:35:22.441735 10408 raft_consensus.cc:2726] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:35:24.888953 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:35:24.889123 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:35:27.412240 10431 raft_consensus.cc:482] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4a331de1488e4a0986b490917efea000)
I20260811 01:35:27.412535 10431 raft_consensus.cc:504] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:35:27.413255 10431 leader_election.cc:288] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:35:27.414413 6702 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
W20260811 01:35:27.414686 6699 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
I20260811 01:35:27.414814 6699 leader_election.cc:302] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:35:27.415096 10431 raft_consensus.cc:2726] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:35:27.864418 10435 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:35:27.864666 10435 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:35:27.865379 10435 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:35:27.866895 9214 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260811 01:35:27.871292 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:35:27.871513 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:35:27.871599 9214 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:35:27.871814 10435 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 01:35:28.750694 10450 raft_consensus.cc:482] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:35:28.750976 10450 raft_consensus.cc:504] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:35:28.751775 10450 leader_election.cc:288] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:35:28.752779 6923 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
W20260811 01:35:28.752985 6920 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
I20260811 01:35:28.753155 6920 leader_election.cc:302] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:35:28.753446 10450 raft_consensus.cc:2726] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:35:29.892704 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:35:29.892951 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:35:33.336328 10495 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:35:33.336553 10495 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:35:33.337285 10495 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:35:33.338743 9214 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 01:35:33.340232 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:35:33.340409 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:35:33.340480 9214 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:35:33.340713 10495 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 01:35:34.071696 10518 raft_consensus.cc:482] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4a331de1488e4a0986b490917efea000)
I20260811 01:35:34.071996 10518 raft_consensus.cc:504] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:35:34.072845 10518 leader_election.cc:288] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:35:34.073945 6702 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
W20260811 01:35:34.074174 6699 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
I20260811 01:35:34.074311 6699 leader_election.cc:302] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:35:34.074563 10518 raft_consensus.cc:2726] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:35:34.896096 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:35:34.896315 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:35:35.684481 10546 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:35:35.684705 10546 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:35:35.685544 10546 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:35:35.689286 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:35:35.689500 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:35:35.689602 9214 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:35:35.689829 10546 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 01:35:37.776512 10572 raft_consensus.cc:482] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:35:37.776738 10572 raft_consensus.cc:504] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:35:37.777632 10572 leader_election.cc:288] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:35:37.778828 6920 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
W20260811 01:35:37.779068 6923 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
I20260811 01:35:37.779256 6923 leader_election.cc:302] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:35:37.779533 10572 raft_consensus.cc:2726] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:35:37.863092 10576 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:35:37.863381 10576 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:35:37.864094 10576 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:35:37.866678 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:35:37.866916 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:35:37.867064 9214 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:35:37.867352 10576 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 01:35:38.867309 10588 raft_consensus.cc:482] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:35:38.867589 10588 raft_consensus.cc:504] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } }
I20260811 01:35:38.868247 10588 leader_election.cc:288] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:35:38.869269 6920 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
W20260811 01:35:38.869509 6923 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
I20260811 01:35:38.869609 6923 leader_election.cc:302] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:35:38.869845 10588 raft_consensus.cc:2726] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:35:39.899489 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:35:39.899711 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:35:40.455188 10617 raft_consensus.cc:482] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 992432fc73c94fcc9f2142b27ef725af)
I20260811 01:35:40.455452 10617 raft_consensus.cc:504] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:35:40.456351 10617 leader_election.cc:288] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:35:40.457325 6699 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
W20260811 01:35:40.457528 6702 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
I20260811 01:35:40.457625 6702 leader_election.cc:302] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:35:40.457825 10617 raft_consensus.cc:2726] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:35:41.147691 10627 raft_consensus.cc:482] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:35:41.147893 10627 raft_consensus.cc:504] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:35:41.148785 10627 leader_election.cc:288] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:35:41.149799 6923 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
W20260811 01:35:41.149991 6920 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
I20260811 01:35:41.150157 6920 leader_election.cc:302] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:35:41.150395 10627 raft_consensus.cc:2726] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:35:44.905642 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:35:44.905807 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:35:45.090279 10671 raft_consensus.cc:482] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 992432fc73c94fcc9f2142b27ef725af)
I20260811 01:35:45.090543 10671 raft_consensus.cc:504] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:35:45.091253 10671 leader_election.cc:288] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:35:45.092326 6702 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
W20260811 01:35:45.092573 6699 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
I20260811 01:35:45.092676 6699 leader_election.cc:302] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:35:45.092902 10671 raft_consensus.cc:2726] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:35:45.910524 10677 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:35:45.910871 10677 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:35:45.911685 10677 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:35:45.913096 9214 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111) [suppressed 5 similar messages]
W20260811 01:35:45.915307 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:35:45.915608 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:35:45.915692 9214 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:35:45.916004 10677 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 01:35:46.933254 10699 raft_consensus.cc:482] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:35:46.933478 10699 raft_consensus.cc:504] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:35:46.934336 10699 leader_election.cc:288] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:35:46.935417 6923 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
W20260811 01:35:46.935647 6920 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
I20260811 01:35:46.935834 6920 leader_election.cc:302] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:35:46.936098 10699 raft_consensus.cc:2726] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:35:49.909070 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:35:49.909253 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:35:53.104926 10769 raft_consensus.cc:482] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4a331de1488e4a0986b490917efea000)
I20260811 01:35:53.105230 10769 raft_consensus.cc:504] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:35:53.106041 10769 leader_election.cc:288] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:35:53.107098 6699 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
W20260811 01:35:53.107272 6702 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
I20260811 01:35:53.107398 6702 leader_election.cc:302] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:35:53.107668 10769 raft_consensus.cc:2726] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:35:54.912420 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:35:54.912617 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 01:35:57.484951 6654 heartbeater.cc:644] Failed to heartbeat to 127.5.107.252:45519 (564 consecutive failures): Network error: Failed to ping master at 127.5.107.252:45519: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:35:57.556186 10813 raft_consensus.cc:482] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 992432fc73c94fcc9f2142b27ef725af)
I20260811 01:35:57.556398 10813 raft_consensus.cc:504] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:35:57.557163 10813 leader_election.cc:288] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:35:57.558226 6699 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
W20260811 01:35:57.558409 6702 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
I20260811 01:35:57.558537 6702 leader_election.cc:302] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:35:57.558781 10813 raft_consensus.cc:2726] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:35:57.709041 10814 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:35:57.709265 10814 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:35:57.710103 10814 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:35:57.711555 9214 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 01:35:57.712990 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:35:57.713160 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:35:57.713231 9214 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:35:57.713470 10814 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260811 01:35:57.811354 6896 heartbeater.cc:644] Failed to heartbeat to 127.5.107.253:36427 (550 consecutive failures): Network error: Failed to ping master at 127.5.107.253:36427: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:35:57.811348 7113 heartbeater.cc:644] Failed to heartbeat to 127.5.107.253:36427 (550 consecutive failures): Network error: Failed to ping master at 127.5.107.253:36427: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
I20260811 01:35:58.114089 10826 raft_consensus.cc:482] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:35:58.114320 10826 raft_consensus.cc:504] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } }
I20260811 01:35:58.115164 10826 leader_election.cc:288] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:35:58.116246 6923 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
W20260811 01:35:58.116494 6920 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
I20260811 01:35:58.116674 6920 leader_election.cc:302] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:35:58.116955 10826 raft_consensus.cc:2726] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:35:59.836248 9218 maintenance_manager.cc:419] P 61c27cf3d4c846748ae6752e0fa652b9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 828 bytes on disk
I20260811 01:35:59.842006 9217 maintenance_manager.cc:643] P 61c27cf3d4c846748ae6752e0fa652b9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.005s sys 0.000s Metrics: {"dirs.queue_time_us":431,"dirs.run_cpu_time_us":142,"dirs.run_wall_time_us":143,"thread_start_us":539,"threads_started":2,"wal-append.queue_time_us":372}
I20260811 01:35:59.892845 10850 raft_consensus.cc:482] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:35:59.893038 10850 raft_consensus.cc:504] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:35:59.893673 10850 leader_election.cc:288] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:35:59.894701 6920 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
W20260811 01:35:59.894847 6923 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
I20260811 01:35:59.895035 6923 leader_election.cc:302] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:35:59.895321 10850 raft_consensus.cc:2726] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:35:59.915377 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:35:59.915586 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:36:02.702998 10882 raft_consensus.cc:482] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4a331de1488e4a0986b490917efea000)
I20260811 01:36:02.703277 10882 raft_consensus.cc:504] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:36:02.704015 10882 leader_election.cc:288] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:36:02.705044 6702 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
W20260811 01:36:02.705308 6699 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
I20260811 01:36:02.705432 6699 leader_election.cc:302] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:36:02.705731 10882 raft_consensus.cc:2726] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:36:04.913961 10902 raft_consensus.cc:482] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:36:04.914247 10902 raft_consensus.cc:504] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } }
I20260811 01:36:04.915040 10902 leader_election.cc:288] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:36:04.916101 6920 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
W20260811 01:36:04.916277 6923 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
I20260811 01:36:04.916420 6923 leader_election.cc:302] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:36:04.916688 10902 raft_consensus.cc:2726] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:36:04.918584 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:36:04.918776 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:36:08.866101 10960 raft_consensus.cc:482] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:36:08.866364 10960 raft_consensus.cc:504] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:36:08.867044 10960 leader_election.cc:288] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:36:08.868059 6923 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
W20260811 01:36:08.868294 6920 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
I20260811 01:36:08.868479 6920 leader_election.cc:302] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:36:08.868722 10960 raft_consensus.cc:2726] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:36:09.244288 10963 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:36:09.244573 10963 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61c27cf3d4c846748ae6752e0fa652b9" member_type: VOTER last_known_addr { host: "127.5.107.254" port: 33755 } } peers { permanent_uuid: "8970a89e5fd744008240ecd668272a4a" member_type: VOTER last_known_addr { host: "127.5.107.253" port: 36427 } } peers { permanent_uuid: "c6f417c29b464762944a721d477b2b13" member_type: VOTER last_known_addr { host: "127.5.107.252" port: 45519 } }
I20260811 01:36:09.245296 10963 leader_election.cc:288] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427), c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519)
W20260811 01:36:09.246879 9214 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 01:36:09.248486 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8970a89e5fd744008240ecd668272a4a (127.5.107.253:36427): Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111)
W20260811 01:36:09.248709 9214 leader_election.cc:334] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c6f417c29b464762944a721d477b2b13 (127.5.107.252:45519): Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
I20260811 01:36:09.248791 9214 leader_election.cc:302] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61c27cf3d4c846748ae6752e0fa652b9; no voters: 8970a89e5fd744008240ecd668272a4a, c6f417c29b464762944a721d477b2b13
I20260811 01:36:09.249135 10963 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61c27cf3d4c846748ae6752e0fa652b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 01:36:09.921983 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:36:09.922389 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:36:12.954078 11010 raft_consensus.cc:482] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:36:12.954325 11010 raft_consensus.cc:504] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:36:12.955071 11010 leader_election.cc:288] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:36:12.956066 6920 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
W20260811 01:36:12.956291 6923 leader_election.cc:341] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 01ab509f88494b5f998fd1705c5d0f59
I20260811 01:36:12.956405 6923 leader_election.cc:302] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:36:12.956705 11010 raft_consensus.cc:2726] T 01ab509f88494b5f998fd1705c5d0f59 P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:36:14.925983 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:36:14.926229 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:36:15.873456 11043 raft_consensus.cc:482] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:36:15.873696 11043 raft_consensus.cc:504] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } }
I20260811 01:36:15.874653 11043 leader_election.cc:288] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971), 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339)
W20260811 01:36:15.875528 6920 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 10a02c605db44ec69c67a34f53c2a055 (127.5.107.194:32971): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
W20260811 01:36:15.875825 6923 leader_election.cc:341] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 96e04578bb164dd48866d5925b000eff
I20260811 01:36:15.875959 6923 leader_election.cc:302] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 992432fc73c94fcc9f2142b27ef725af; no voters: 10a02c605db44ec69c67a34f53c2a055, 4a331de1488e4a0986b490917efea000
I20260811 01:36:15.876169 11043 raft_consensus.cc:2726] T 96e04578bb164dd48866d5925b000eff P 992432fc73c94fcc9f2142b27ef725af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:36:16.312733 11050 raft_consensus.cc:482] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 992432fc73c94fcc9f2142b27ef725af)
I20260811 01:36:16.312963 11050 raft_consensus.cc:504] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } }
I20260811 01:36:16.313908 11050 leader_election.cc:288] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:36:16.314806 6702 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
W20260811 01:36:16.315022 6699 leader_election.cc:341] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: 1cb26d431b24475a8cf62325267b0a3d
I20260811 01:36:16.315158 6699 leader_election.cc:302] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:36:16.315407 11050 raft_consensus.cc:2726] T 1cb26d431b24475a8cf62325267b0a3d P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 01:36:19.931012 9186 master_runner.cc:186] Error getting master registration for 127.5.107.253:36427: Network error: Client connection negotiation failed: client connection to 127.5.107.253:36427: connect: Connection refused (error 111),
I20260811 01:36:19.931234 9186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:36:22.132285 11117 raft_consensus.cc:482] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4a331de1488e4a0986b490917efea000)
I20260811 01:36:22.132524 11117 raft_consensus.cc:504] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a331de1488e4a0986b490917efea000" member_type: VOTER last_known_addr { host: "127.5.107.193" port: 43339 } } peers { permanent_uuid: "992432fc73c94fcc9f2142b27ef725af" member_type: VOTER last_known_addr { host: "127.5.107.195" port: 39433 } } peers { permanent_uuid: "10a02c605db44ec69c67a34f53c2a055" member_type: VOTER last_known_addr { host: "127.5.107.194" port: 32971 } }
I20260811 01:36:22.133212 11117 leader_election.cc:288] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339), 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433)
W20260811 01:36:22.134331 6702 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a331de1488e4a0986b490917efea000 (127.5.107.193:43339): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
W20260811 01:36:22.134547 6699 leader_election.cc:341] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 992432fc73c94fcc9f2142b27ef725af (127.5.107.195:39433): Not found: Tablet not found: a6bc3c3d5d904830bfda3966e47305da
I20260811 01:36:22.134690 6699 leader_election.cc:302] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10a02c605db44ec69c67a34f53c2a055; no voters: 4a331de1488e4a0986b490917efea000, 992432fc73c94fcc9f2142b27ef725af
I20260811 01:36:22.134982 11117 raft_consensus.cc:2726] T a6bc3c3d5d904830bfda3966e47305da P 10a02c605db44ec69c67a34f53c2a055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260811 01:36:22.453156 7131 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
W20260811 01:36:22.453532 7131 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
W20260811 01:36:22.453670 7131 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
F20260811 01:36:22.464609 7175 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260811 01:36:22.464732 7158 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_34b4086ff0f64632adbac74648d9874a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
*** Check failure stack trace: ***
*** Aborted at 1786412182 (unix time) try "date -d @1786412182" if you are using GNU date ***
F20260811 01:36:22.464732 7158 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_34b4086ff0f64632adbac74648d9874a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.465938 7141 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9f9bb4e0b22447c79a5597bc7db4eec8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260811 01:36:22.464732 7158 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_34b4086ff0f64632adbac74648d9874a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.465938 7141 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9f9bb4e0b22447c79a5597bc7db4eec8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.467346 7182 master-stress-test.cc:392] Check failed: _s.ok() Bad status: F20260811 01:36:22.467336 7143 master-stress-test.cc:392] Timed out: ReplaceTablet timed o: after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111) F*** Check failure stack trace: ***
F20260811 01:36:22.464732 7158 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_34b4086ff0f64632adbac74648d9874a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.465938 7141 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9f9bb4e0b22447c79a5597bc7db4eec8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.467346 7182 master-stress-test.cc:392] Check failed: _s.ok() Bad status: F20260811 01:36:22.467336 7143 master-stress-test.cc:392] Timed out: ReplaceTablet timed o: after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111) F2026Timed out: Error creating table default.Table_e6b7b1e823f84e859b8956dc8c474b29 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)0811 01:36:22.467307 7155 master-stress-test.cc:392] Check failed: _s.ok() Bad status:
*** Check failure stack trace: ***
F20260811 01:36:22.464732 7158 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_34b4086ff0f64632adbac74648d9874a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.465938 7141 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9f9bb4e0b22447c79a5597bc7db4eec8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.467346 7182 master-stress-test.cc:392] Check failed: _s.ok() Bad status: F20260811 01:36:22.467336 7143 master-stress-test.cc:392] Timed out: ReplaceTablet timed o: after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111) F2026Timed out: Error creating table default.Table_e6b7b1e823f84e859b8956dc8c474b29 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)0811 01:36:22.467307 7155 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_071a47d589c9407aa47c9d668c44cc23 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260811 01:36:22.464732 7158 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_34b4086ff0f64632adbac74648d9874a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.465938 7141 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9f9bb4e0b22447c79a5597bc7db4eec8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.467346 7182 master-stress-test.cc:392] Check failed: _s.ok() Bad status: F20260811 01:36:22.467336 7143 master-stress-test.cc:392] Timed out: ReplaceTablet timed o: after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111) F2026Timed out: Error creating table default.Table_e6b7b1e823f84e859b8956dc8c474b29 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)0811 01:36:22.467307 7155 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_071a47d589c9407aa47c9d668c44cc23 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.468583 7157 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4fad8fd88a6444b2a3dbc99d87835413 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260811 01:36:22.464732 7158 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_34b4086ff0f64632adbac74648d9874a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.465938 7141 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9f9bb4e0b22447c79a5597bc7db4eec8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.467346 7182 master-stress-test.cc:392] Check failed: _s.ok() Bad status: F20260811 01:36:22.467336 7143 master-stress-test.cc:392] Timed out: ReplaceTablet timed o: after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111) F2026Timed out: Error creating table default.Table_e6b7b1e823f84e859b8956dc8c474b29 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)0811 01:36:22.467307 7155 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_071a47d589c9407aa47c9d668c44cc23 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.468583 7157 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4fad8fd88a6444b2a3dbc99d87835413 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.468806 7162 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260811 01:36:22.464732 7158 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_34b4086ff0f64632adbac74648d9874a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.465938 7141 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9f9bb4e0b22447c79a5597bc7db4eec8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.467346 7182 master-stress-test.cc:392] Check failed: _s.ok() Bad status: F20260811 01:36:22.467336 7143 master-stress-test.cc:392] Timed out: ReplaceTablet timed o: after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111) F2026Timed out: Error creating table default.Table_e6b7b1e823f84e859b8956dc8c474b29 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)0811 01:36:22.467307 7155 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_071a47d589c9407aa47c9d668c44cc23 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.468583 7157 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4fad8fd88a6444b2a3dbc99d87835413 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.468806 7162 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.469882 7146 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0bf1f8de8ab54285b6291b6c6e30b647 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260811 01:36:22.464732 7158 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_34b4086ff0f64632adbac74648d9874a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.465938 7141 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9f9bb4e0b22447c79a5597bc7db4eec8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.467346 7182 master-stress-test.cc:392] Check failed: _s.ok() Bad status: F20260811 01:36:22.467336 7143 master-stress-test.cc:392] Timed out: ReplaceTablet timed o: after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111) F2026Timed out: Error creating table default.Table_e6b7b1e823f84e859b8956dc8c474b29 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)0811 01:36:22.467307 7155 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_071a47d589c9407aa47c9d668c44cc23 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.468583 7157 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4fad8fd88a6444b2a3dbc99d87835413 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.468806 7162 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.469882 7146 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0bf1f8de8ab54285b6291b6c6e30b647 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.469946 7161 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260811 01:36:22.464732 7158 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_34b4086ff0f64632adbac74648d9874a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.465938 7141 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9f9bb4e0b22447c79a5597bc7db4eec8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.467346 7182 master-stress-test.cc:392] Check failed: _s.ok() Bad status: F20260811 01:36:22.467336 7143 master-stress-test.cc:392] Timed out: ReplaceTablet timed o: after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111) F2026Timed out: Error creating table default.Table_e6b7b1e823f84e859b8956dc8c474b29 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)0811 01:36:22.467307 7155 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_071a47d589c9407aa47c9d668c44cc23 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.468583 7157 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4fad8fd88a6444b2a3dbc99d87835413 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.468806 7162 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.469882 7146 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0bf1f8de8ab54285b6291b6c6e30b647 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.469946 7161 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.470290 7145 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_977e759aa4644f0091ecc8c4613cffc2 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260811 01:36:22.464732 7158 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_34b4086ff0f64632adbac74648d9874a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.465938 7141 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9f9bb4e0b22447c79a5597bc7db4eec8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.467346 7182 master-stress-test.cc:392] Check failed: _s.ok() Bad status: F20260811 01:36:22.467336 7143 master-stress-test.cc:392] Timed out: ReplaceTablet timed o: after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111) F2026Timed out: Error creating table default.Table_e6b7b1e823f84e859b8956dc8c474b29 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)0811 01:36:22.467307 7155 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_071a47d589c9407aa47c9d668c44cc23 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.468583 7157 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4fad8fd88a6444b2a3dbc99d87835413 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.468806 7162 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.469882 7146 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0bf1f8de8ab54285b6291b6c6e30b647 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.469946 7161 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.470290 7145 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_977e759aa4644f0091ecc8c4613cffc2 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.470350 7151 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5e80cfaf385a44328ab0113196274005 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260811 01:36:22.464732 7158 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_34b4086ff0f64632adbac74648d9874a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.465938 7141 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9f9bb4e0b22447c79a5597bc7db4eec8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.467346 7182 master-stress-test.cc:392] Check failed: _s.ok() Bad status: F20260811 01:36:22.467336 7143 master-stress-test.cc:392] Timed out: ReplaceTablet timed o: after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111) F2026Timed out: Error creating table default.Table_e6b7b1e823f84e859b8956dc8c474b29 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)0811 01:36:22.467307 7155 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_071a47d589c9407aa47c9d668c44cc23 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.468583 7157 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4fad8fd88a6444b2a3dbc99d87835413 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.468806 7162 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.469882 7146 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0bf1f8de8ab54285b6291b6c6e30b647 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.469946 7161 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.470290 7145 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_977e759aa4644f0091ecc8c4613cffc2 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.470350 7151 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5e80cfaf385a44328ab0113196274005 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.471561 7180 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
*** Check failure stack trace: ***
PC: @ 0x0 (unknown)
*** SIGABRT (@0x3e8000015af) received by PID 5551 (TID 0x7f63e7edb700) from PID 5551; stack trace: ***
F20260811 01:36:22.464732 7158 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_34b4086ff0f64632adbac74648d9874a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.465938 7141 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9f9bb4e0b22447c79a5597bc7db4eec8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.467346 7182 master-stress-test.cc:392] Check failed: _s.ok() Bad status: F20260811 01:36:22.467336 7143 master-stress-test.cc:392] Timed out: ReplaceTablet timed o: after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111) F2026Timed out: Error creating table default.Table_e6b7b1e823f84e859b8956dc8c474b29 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)0811 01:36:22.467307 7155 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_071a47d589c9407aa47c9d668c44cc23 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.468583 7157 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4fad8fd88a6444b2a3dbc99d87835413 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.468806 7162 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.469882 7146 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0bf1f8de8ab54285b6291b6c6e30b647 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.469946 7161 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.470290 7145 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_977e759aa4644f0091ecc8c4613cffc2 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.470350 7151 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5e80cfaf385a44328ab0113196274005 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.471561 7180 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.472728 7181 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260811 01:36:22.464732 7158 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_34b4086ff0f64632adbac74648d9874a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.465938 7141 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9f9bb4e0b22447c79a5597bc7db4eec8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.467346 7182 master-stress-test.cc:392] Check failed: _s.ok() Bad status: F20260811 01:36:22.467336 7143 master-stress-test.cc:392] Timed out: ReplaceTablet timed o: after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111) F2026Timed out: Error creating table default.Table_e6b7b1e823f84e859b8956dc8c474b29 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)0811 01:36:22.467307 7155 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_071a47d589c9407aa47c9d668c44cc23 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.468583 7157 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4fad8fd88a6444b2a3dbc99d87835413 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.468806 7162 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.469882 7146 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0bf1f8de8ab54285b6291b6c6e30b647 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.469946 7161 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.470290 7145 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_977e759aa4644f0091ecc8c4613cffc2 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.470350 7151 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5e80cfaf385a44328ab0113196274005 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.471561 7180 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.472728 7181 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.472863 7176 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260811 01:36:22.464732 7158 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_34b4086ff0f64632adbac74648d9874a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.465938 7141 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9f9bb4e0b22447c79a5597bc7db4eec8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.467346 7182 master-stress-test.cc:392] Check failed: _s.ok() Bad status: F20260811 01:36:22.467336 7143 master-stress-test.cc:392] Timed out: ReplaceTablet timed o: after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111) F2026Timed out: Error creating table default.Table_e6b7b1e823f84e859b8956dc8c474b29 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)0811 01:36:22.467307 7155 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_071a47d589c9407aa47c9d668c44cc23 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.468583 7157 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4fad8fd88a6444b2a3dbc99d87835413 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.468806 7162 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.469882 7146 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0bf1f8de8ab54285b6291b6c6e30b647 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.469946 7161 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.470290 7145 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_977e759aa4644f0091ecc8c4613cffc2 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.470350 7151 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5e80cfaf385a44328ab0113196274005 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.471561 7180 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.472728 7181 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.472863 7176 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.473016 7144 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_16925b327c134e08987029291b57034c on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
*** Check failure stack trace: ***
@ 0x7f63fa568a46 google::(anonymous namespace)::FailureSignalHandler() at ??:0
F20260811 01:36:22.464732 7158 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_34b4086ff0f64632adbac74648d9874a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.465938 7141 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9f9bb4e0b22447c79a5597bc7db4eec8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.467346 7182 master-stress-test.cc:392] Check failed: _s.ok() Bad status: F20260811 01:36:22.467336 7143 master-stress-test.cc:392] Timed out: ReplaceTablet timed o: after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111) F2026Timed out: Error creating table default.Table_e6b7b1e823f84e859b8956dc8c474b29 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)0811 01:36:22.467307 7155 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_071a47d589c9407aa47c9d668c44cc23 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.468583 7157 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4fad8fd88a6444b2a3dbc99d87835413 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.468806 7162 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.469882 7146 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0bf1f8de8ab54285b6291b6c6e30b647 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.469946 7161 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.470290 7145 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_977e759aa4644f0091ecc8c4613cffc2 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.470350 7151 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5e80cfaf385a44328ab0113196274005 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.471561 7180 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.472728 7181 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.472863 7176 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.473016 7144 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_16925b327c134e08987029291b57034c on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.474499 7163 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260811 01:36:22.464732 7158 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_34b4086ff0f64632adbac74648d9874a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.465938 7141 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9f9bb4e0b22447c79a5597bc7db4eec8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.467346 7182 master-stress-test.cc:392] Check failed: _s.ok() Bad status: F20260811 01:36:22.467336 7143 master-stress-test.cc:392] Timed out: ReplaceTablet timed o: after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111) F2026Timed out: Error creating table default.Table_e6b7b1e823f84e859b8956dc8c474b29 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)0811 01:36:22.467307 7155 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_071a47d589c9407aa47c9d668c44cc23 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.468583 7157 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4fad8fd88a6444b2a3dbc99d87835413 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.468806 7162 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.469882 7146 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0bf1f8de8ab54285b6291b6c6e30b647 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.469946 7161 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.470290 7145 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_977e759aa4644f0091ecc8c4613cffc2 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.470350 7151 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5e80cfaf385a44328ab0113196274005 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.471561 7180 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.472728 7181 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.472863 7176 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.473016 7144 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_16925b327c134e08987029291b57034c on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.474499 7163 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)F20260811 01:36:22.474638 7142 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1354313d091b46d88f34c2ccd44bf8e0 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.5.107.254:33755,127.5.107.253:36427,127.5.107.252:45519, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.5.107.252:45519: connect: Connection refused (error 111)
*** Check failure stack trace: ***
@ 0x7f63fe6aa980 (unknown) at ??:0
@ 0x7f63f7896e87 gsignal at ??:0
@ 0x7f63f78987f1 abort at ??:0
@ 0x7f63fa55a9ad google::LogMessage::Fail() at ??:0
@ 0x7f63fa55d88a google::LogMessage::SendToLog() at ??:0
@ 0x7f63fa55a4de google::LogMessage::Flush() at ??:0
@ 0x7f63fa55ba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x403d42 kudu::MasterStressTest::CreateTableThread() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/master-stress-test.cc:298
@ 0x444b44 std::__invoke_impl<>() at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/bits/invoke.h:73
@ 0x4449fd std::__invoke<>() at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/bits/invoke.h:95
@ 0x4449ba _ZNSt6thread8_InvokerISt5tupleIJMN4kudu16MasterStressTestEFvvEPNS2_26MasterStressTest_Test_TestEEEE9_M_invokeIJLm0ELm1EEEEDTclsr3stdE8__invokespcl10_S_declvalIXT_EEEEESt12_Index_tupleIJXspT_EEE at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/thread:244
@ 0x44496e std::thread::_Invoker<>::operator()() at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/thread:253
@ 0x4446a9 std::thread::_State_impl<>::_M_run() at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/thread:196
@ 0x7f6402fe6a4f execute_native_thread_routine at ??:0
@ 0x7f63fe69f6db start_thread at ??:0
@ 0x7f63f797961f clone at ??:0