Note: This is test shard 3 of 3.
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from CatalogManagerConfigurations/MasterStressTest
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/2
2026-05-01T13:52:37Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-05-01T13:52:37Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-task4ezAfy/thirdparty/src/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-task4ezAfy/thirdparty/src/hadoop-3.4.1/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/security.properties
2026-05-01 13:52:45: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-task4ezAfy/thirdparty/src/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-task4ezAfy/thirdparty/src/hadoop-3.4.1/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260501 13:52:53.491376 420 external_mini_cluster.cc:1366] Running /tmp/dist-test-task4ezAfy/build/asan/bin/kudu
/tmp/dist-test-task4ezAfy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.62:44705
--webserver_interface=127.0.105.62
--webserver_port=0
--builtin_ntp_servers=127.0.105.20:38455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729
--hive_metastore_uris=thrift://127.0.0.1:33681
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260501 13:52:53.826717 705 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260501 13:52:53.827091 705 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260501 13:52:53.827235 705 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260501 13:52:53.836232 705 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260501 13:52:53.836365 705 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260501 13:52:53.836480 705 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260501 13:52:53.836565 705 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260501 13:52:53.847087 705 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.105.20:38455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:33681
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.62:44705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.105.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision aa7cc0aa9c559b87f264235eaadd03ec492277e4
build type FASTDEBUG
built by None at 01 May 2026 13:43:23 UTC on bdcb31816ec0
build id 11694
ASAN enabled
I20260501 13:52:53.848800 705 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260501 13:52:53.850615 705 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260501 13:52:53.861562 710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260501 13:52:53.861796 711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260501 13:52:53.862013 713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260501 13:52:53.862982 705 server_base.cc:1061] running on GCE node
I20260501 13:52:53.863806 705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260501 13:52:53.865268 705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260501 13:52:53.866520 705 hybrid_clock.cc:648] HybridClock initialized: now 1777643573866434 us; error 66 us; skew 500 ppm
I20260501 13:52:53.866905 705 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260501 13:52:53.869009 705 webserver.cc:492] Webserver started at http://127.0.105.62:45017/ using document root <none> and password file <none>
I20260501 13:52:53.869747 705 fs_manager.cc:362] Metadata directory not provided
I20260501 13:52:53.869874 705 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260501 13:52:53.870203 705 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260501 13:52:53.872586 705 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data/instance:
uuid: "5fa23b4772a944c0b0279eabd7476196"
format_stamp: "Formatted at 2026-05-01 13:52:53 on dist-test-slave-qlwv"
I20260501 13:52:53.873814 705 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/wal/instance:
uuid: "5fa23b4772a944c0b0279eabd7476196"
format_stamp: "Formatted at 2026-05-01 13:52:53 on dist-test-slave-qlwv"
I20260501 13:52:53.879432 705 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260501 13:52:53.882820 719 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260501 13:52:53.884191 705 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260501 13:52:53.884387 705 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data,/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/wal
uuid: "5fa23b4772a944c0b0279eabd7476196"
format_stamp: "Formatted at 2026-05-01 13:52:53 on dist-test-slave-qlwv"
I20260501 13:52:53.884606 705 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260501 13:52:53.899001 705 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260501 13:52:53.899915 705 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260501 13:52:53.900177 705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260501 13:52:53.922770 705 rpc_server.cc:307] RPC server started. Bound to: 127.0.105.62:44705
I20260501 13:52:53.922793 770 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.105.62:44705 every 8 connection(s)
I20260501 13:52:53.924398 705 server_base.cc:1193] Dumped server information to /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data/info.pb
I20260501 13:52:53.928534 420 external_mini_cluster.cc:1428] Started /tmp/dist-test-task4ezAfy/build/asan/bin/kudu as pid 705
I20260501 13:52:53.928829 420 external_mini_cluster.cc:1442] Reading /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/wal/instance
I20260501 13:52:53.929214 771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260501 13:52:53.931715 420 external_mini_cluster.cc:1366] Running /tmp/dist-test-task4ezAfy/build/asan/bin/kudu
/tmp/dist-test-task4ezAfy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.61:33209
--webserver_interface=127.0.105.61
--webserver_port=0
--builtin_ntp_servers=127.0.105.20:38455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729
--hive_metastore_uris=thrift://127.0.0.1:33681
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260501 13:52:53.934942 771 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } has no permanent_uuid. Determining permanent_uuid...
I20260501 13:52:53.955736 771 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } has no permanent_uuid. Determining permanent_uuid...
W20260501 13:52:53.957476 723 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:52:53.959527 771 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
I20260501 13:52:54.008989 771 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } attempt: 1
W20260501 13:52:54.011000 771 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
I20260501 13:52:54.079378 771 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } attempt: 2
W20260501 13:52:54.081334 771 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
I20260501 13:52:54.172622 771 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } attempt: 3
W20260501 13:52:54.174542 771 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:52:54.283519 772 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260501 13:52:54.283882 772 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260501 13:52:54.284029 772 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260501 13:52:54.293368 772 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260501 13:52:54.293526 772 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260501 13:52:54.293640 772 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260501 13:52:54.293722 772 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260501 13:52:54.304756 772 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.105.20:38455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:33681
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.61:33209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.105.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision aa7cc0aa9c559b87f264235eaadd03ec492277e4
build type FASTDEBUG
built by None at 01 May 2026 13:43:23 UTC on bdcb31816ec0
build id 11694
ASAN enabled
I20260501 13:52:54.306427 772 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260501 13:52:54.308175 772 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260501 13:52:54.315986 781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260501 13:52:54.316069 780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260501 13:52:54.316865 783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260501 13:52:54.317800 771 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } attempt: 4
I20260501 13:52:54.318042 772 server_base.cc:1061] running on GCE node
I20260501 13:52:54.319680 772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
W20260501 13:52:54.320000 771 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
I20260501 13:52:54.321374 772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260501 13:52:54.322635 772 hybrid_clock.cc:648] HybridClock initialized: now 1777643574322590 us; error 43 us; skew 500 ppm
I20260501 13:52:54.323107 772 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260501 13:52:54.326063 772 webserver.cc:492] Webserver started at http://127.0.105.61:45377/ using document root <none> and password file <none>
I20260501 13:52:54.326845 772 fs_manager.cc:362] Metadata directory not provided
I20260501 13:52:54.326936 772 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260501 13:52:54.327342 772 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260501 13:52:54.331663 772 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-1/data/instance:
uuid: "84a39a5f167243ff9bfe07189aa881ca"
format_stamp: "Formatted at 2026-05-01 13:52:54 on dist-test-slave-qlwv"
I20260501 13:52:54.332448 772 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-1/wal/instance:
uuid: "84a39a5f167243ff9bfe07189aa881ca"
format_stamp: "Formatted at 2026-05-01 13:52:54 on dist-test-slave-qlwv"
I20260501 13:52:54.337699 772 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260501 13:52:54.341094 789 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260501 13:52:54.342466 772 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260501 13:52:54.342641 772 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-1/data,/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-1/wal
uuid: "84a39a5f167243ff9bfe07189aa881ca"
format_stamp: "Formatted at 2026-05-01 13:52:54 on dist-test-slave-qlwv"
I20260501 13:52:54.342852 772 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260501 13:52:54.357650 772 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260501 13:52:54.358570 772 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260501 13:52:54.358875 772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260501 13:52:54.381697 840 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.105.61:33209 every 8 connection(s)
I20260501 13:52:54.381691 772 rpc_server.cc:307] RPC server started. Bound to: 127.0.105.61:33209
I20260501 13:52:54.383579 772 server_base.cc:1193] Dumped server information to /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-1/data/info.pb
I20260501 13:52:54.387903 841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260501 13:52:54.390770 420 external_mini_cluster.cc:1428] Started /tmp/dist-test-task4ezAfy/build/asan/bin/kudu as pid 772
I20260501 13:52:54.390974 420 external_mini_cluster.cc:1442] Reading /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-1/wal/instance
I20260501 13:52:54.392056 420 external_mini_cluster.cc:1366] Running /tmp/dist-test-task4ezAfy/build/asan/bin/kudu
/tmp/dist-test-task4ezAfy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.60:39729
--webserver_interface=127.0.105.60
--webserver_port=0
--builtin_ntp_servers=127.0.105.20:38455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729
--hive_metastore_uris=thrift://127.0.0.1:33681
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260501 13:52:54.394006 841 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } has no permanent_uuid. Determining permanent_uuid...
I20260501 13:52:54.408390 841 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } has no permanent_uuid. Determining permanent_uuid...
I20260501 13:52:54.418725 841 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } has no permanent_uuid. Determining permanent_uuid...
W20260501 13:52:54.419746 790 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
W20260501 13:52:54.421339 841 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.105.60:39729: Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:52:54.470676 841 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } attempt: 1
W20260501 13:52:54.472681 841 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.105.60:39729: Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:52:54.540910 841 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } attempt: 2
W20260501 13:52:54.543368 841 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.105.60:39729: Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:52:54.619309 771 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } attempt: 5
I20260501 13:52:54.626912 771 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } has no permanent_uuid. Determining permanent_uuid...
W20260501 13:52:54.629007 771 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.105.60:39729: Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:52:54.634584 841 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } attempt: 3
W20260501 13:52:54.636396 841 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.105.60:39729: Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:52:54.680227 771 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } attempt: 1
W20260501 13:52:54.682503 771 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.105.60:39729: Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
W20260501 13:52:54.735180 842 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260501 13:52:54.735630 842 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260501 13:52:54.735757 842 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260501 13:52:54.744700 842 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260501 13:52:54.744841 842 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260501 13:52:54.744949 842 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260501 13:52:54.745028 842 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260501 13:52:54.750703 771 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } attempt: 2
W20260501 13:52:54.752573 771 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.105.60:39729: Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:52:54.755708 842 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.105.20:38455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:33681
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.60:39729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.105.60
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision aa7cc0aa9c559b87f264235eaadd03ec492277e4
build type FASTDEBUG
built by None at 01 May 2026 13:43:23 UTC on bdcb31816ec0
build id 11694
ASAN enabled
I20260501 13:52:54.757290 842 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260501 13:52:54.759014 842 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260501 13:52:54.766731 850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260501 13:52:54.767042 851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260501 13:52:54.767858 853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260501 13:52:54.769160 842 server_base.cc:1061] running on GCE node
I20260501 13:52:54.769896 842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260501 13:52:54.771262 842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260501 13:52:54.772460 842 hybrid_clock.cc:648] HybridClock initialized: now 1777643574772412 us; error 41 us; skew 500 ppm
I20260501 13:52:54.772804 842 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260501 13:52:54.774792 842 webserver.cc:492] Webserver started at http://127.0.105.60:43413/ using document root <none> and password file <none>
I20260501 13:52:54.775568 842 fs_manager.cc:362] Metadata directory not provided
I20260501 13:52:54.775671 842 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260501 13:52:54.776017 842 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260501 13:52:54.778800 842 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/data/instance:
uuid: "e47102ddbd4f41c1bf0a2864d97237b7"
format_stamp: "Formatted at 2026-05-01 13:52:54 on dist-test-slave-qlwv"
I20260501 13:52:54.779623 842 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/wal/instance:
uuid: "e47102ddbd4f41c1bf0a2864d97237b7"
format_stamp: "Formatted at 2026-05-01 13:52:54 on dist-test-slave-qlwv"
I20260501 13:52:54.779587 841 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } attempt: 4
W20260501 13:52:54.781247 841 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.105.60:39729: Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:52:54.785403 842 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260501 13:52:54.788623 859 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260501 13:52:54.789942 842 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260501 13:52:54.790091 842 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/data,/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/wal
uuid: "e47102ddbd4f41c1bf0a2864d97237b7"
format_stamp: "Formatted at 2026-05-01 13:52:54 on dist-test-slave-qlwv"
I20260501 13:52:54.790293 842 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260501 13:52:54.814481 842 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260501 13:52:54.815363 842 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260501 13:52:54.815585 842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260501 13:52:54.837527 842 rpc_server.cc:307] RPC server started. Bound to: 127.0.105.60:39729
I20260501 13:52:54.837561 910 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.105.60:39729 every 8 connection(s)
I20260501 13:52:54.839187 842 server_base.cc:1193] Dumped server information to /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/data/info.pb
I20260501 13:52:54.843199 911 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260501 13:52:54.849123 420 external_mini_cluster.cc:1428] Started /tmp/dist-test-task4ezAfy/build/asan/bin/kudu as pid 842
I20260501 13:52:54.849298 420 external_mini_cluster.cc:1442] Reading /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/wal/instance
I20260501 13:52:54.848856 911 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } has no permanent_uuid. Determining permanent_uuid...
I20260501 13:52:54.858771 771 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } attempt: 3
I20260501 13:52:54.863905 911 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } has no permanent_uuid. Determining permanent_uuid...
I20260501 13:52:54.871101 911 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } has no permanent_uuid. Determining permanent_uuid...
I20260501 13:52:54.876609 771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: Bootstrap starting.
I20260501 13:52:54.881732 771 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: Neither blocks nor log segments found. Creating new log.
I20260501 13:52:54.882592 911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7: Bootstrap starting.
I20260501 13:52:54.882825 771 log.cc:826] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: Log is configured to *not* fsync() on all Append() calls
I20260501 13:52:54.885214 771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: No bootstrap required, opened a new log
I20260501 13:52:54.887142 911 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7: Neither blocks nor log segments found. Creating new log.
I20260501 13:52:54.888351 911 log.cc:826] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7: Log is configured to *not* fsync() on all Append() calls
I20260501 13:52:54.890727 911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7: No bootstrap required, opened a new log
I20260501 13:52:54.890964 771 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:52:54.891188 771 raft_consensus.cc:385] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260501 13:52:54.891335 771 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fa23b4772a944c0b0279eabd7476196, State: Initialized, Role: FOLLOWER
I20260501 13:52:54.891875 771 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:52:54.892763 920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:52:54.893038 920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: This master's current role is: FOLLOWER
I20260501 13:52:54.893826 771 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: configured and running, proceeding with master startup.
I20260501 13:52:54.896832 911 raft_consensus.cc:359] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:52:54.897157 911 raft_consensus.cc:385] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260501 13:52:54.897329 911 raft_consensus.cc:740] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47102ddbd4f41c1bf0a2864d97237b7, State: Initialized, Role: FOLLOWER
I20260501 13:52:54.897915 911 consensus_queue.cc:260] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:52:54.898839 926 sys_catalog.cc:455] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:52:54.899115 926 sys_catalog.cc:458] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [sys.catalog]: This master's current role is: FOLLOWER
I20260501 13:52:54.899847 911 sys_catalog.cc:565] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [sys.catalog]: configured and running, proceeding with master startup.
W20260501 13:52:54.903501 725 tablet.cc:2404] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260501 13:52:54.919170 926 raft_consensus.cc:493] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:52:54.919533 926 raft_consensus.cc:515] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:52:54.921751 926 leader_election.cc:290] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5fa23b4772a944c0b0279eabd7476196 (127.0.105.62:44705), 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209)
I20260501 13:52:54.923098 746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fa23b4772a944c0b0279eabd7476196" is_pre_election: true
I20260501 13:52:54.923873 746 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e47102ddbd4f41c1bf0a2864d97237b7 in term 0.
I20260501 13:52:54.911681 705 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: OK, instance_id { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" instance_seqno: 1777643574376113 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260501 13:52:54.924556 705 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:52:54.925154 860 leader_election.cc:304] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196, e47102ddbd4f41c1bf0a2864d97237b7; no voters:
I20260501 13:52:54.925773 926 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260501 13:52:54.925932 926 raft_consensus.cc:493] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260501 13:52:54.926039 926 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 0 FOLLOWER]: Advancing to term 1
W20260501 13:52:54.930852 937 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260501 13:52:54.930981 937 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260501 13:52:54.931704 926 raft_consensus.cc:515] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:52:54.933296 746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fa23b4772a944c0b0279eabd7476196"
I20260501 13:52:54.933501 746 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 0 FOLLOWER]: Advancing to term 1
I20260501 13:52:54.939738 746 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47102ddbd4f41c1bf0a2864d97237b7 in term 1.
I20260501 13:52:54.940527 860 leader_election.cc:304] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196, e47102ddbd4f41c1bf0a2864d97237b7; no voters:
I20260501 13:52:54.946229 926 leader_election.cc:290] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [CANDIDATE]: Term 1 election: Requested vote from peers 5fa23b4772a944c0b0279eabd7476196 (127.0.105.62:44705), 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209)
I20260501 13:52:54.947197 926 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260501 13:52:54.947990 926 raft_consensus.cc:697] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 1 LEADER]: Becoming Leader. State: Replica: e47102ddbd4f41c1bf0a2864d97237b7, State: Running, Role: LEADER
I20260501 13:52:54.961547 926 consensus_queue.cc:237] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:52:54.978250 945 sys_catalog.cc:455] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e47102ddbd4f41c1bf0a2864d97237b7. Latest consensus state: current_term: 1 leader_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:52:54.983680 945 sys_catalog.cc:458] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [sys.catalog]: This master's current role is: LEADER
I20260501 13:52:55.028249 951 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260501 13:52:55.047669 956 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260501 13:52:55.051189 951 catalog_manager.cc:1494] Initializing Kudu cluster ID...
W20260501 13:52:55.067575 865 tablet.cc:2404] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260501 13:52:55.061576 842 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: OK, instance_id { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" instance_seqno: 1777643574376113 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260501 13:52:55.070396 842 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:52:55.080734 746 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 1 FOLLOWER]: Refusing update from remote peer e47102ddbd4f41c1bf0a2864d97237b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260501 13:52:55.081238 841 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } attempt: 5
I20260501 13:52:55.083120 945 consensus_queue.cc:1048] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260501 13:52:55.098062 841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca: Bootstrap starting.
I20260501 13:52:55.111893 841 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca: Neither blocks nor log segments found. Creating new log.
I20260501 13:52:55.115911 841 log.cc:826] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca: Log is configured to *not* fsync() on all Append() calls
I20260501 13:52:55.127802 920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e47102ddbd4f41c1bf0a2864d97237b7. Latest consensus state: current_term: 1 leader_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:52:55.128114 920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: This master's current role is: FOLLOWER
I20260501 13:52:55.143960 945 sys_catalog.cc:455] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:52:55.144223 945 sys_catalog.cc:458] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [sys.catalog]: This master's current role is: LEADER
I20260501 13:52:55.150801 841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca: No bootstrap required, opened a new log
I20260501 13:52:55.151539 951 catalog_manager.cc:1357] Generated new cluster ID: 6424d308ce1d424b8fe7d2c1b812f4f7
I20260501 13:52:55.151638 951 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260501 13:52:55.170415 920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:52:55.170642 920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: This master's current role is: FOLLOWER
I20260501 13:52:55.182333 841 raft_consensus.cc:359] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:52:55.183602 841 raft_consensus.cc:385] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260501 13:52:55.184134 841 raft_consensus.cc:740] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84a39a5f167243ff9bfe07189aa881ca, State: Initialized, Role: FOLLOWER
I20260501 13:52:55.187139 841 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:52:55.192178 964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:52:55.192929 964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [sys.catalog]: This master's current role is: FOLLOWER
I20260501 13:52:55.195168 816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84a39a5f167243ff9bfe07189aa881ca" is_pre_election: true
I20260501 13:52:55.195720 814 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 0 FOLLOWER]: Advancing to term 1
I20260501 13:52:55.196033 815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84a39a5f167243ff9bfe07189aa881ca"
I20260501 13:52:55.200238 814 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 1 FOLLOWER]: Refusing update from remote peer e47102ddbd4f41c1bf0a2864d97237b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260501 13:52:55.195164 841 sys_catalog.cc:565] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [sys.catalog]: configured and running, proceeding with master startup.
I20260501 13:52:55.201215 964 raft_consensus.cc:493] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:52:55.201159 926 consensus_queue.cc:1048] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260501 13:52:55.201506 964 raft_consensus.cc:515] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:52:55.206238 886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "84a39a5f167243ff9bfe07189aa881ca" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" is_pre_election: true
I20260501 13:52:55.207116 746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "84a39a5f167243ff9bfe07189aa881ca" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fa23b4772a944c0b0279eabd7476196" is_pre_election: true
I20260501 13:52:55.208079 790 leader_election.cc:304] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84a39a5f167243ff9bfe07189aa881ca; no voters: 5fa23b4772a944c0b0279eabd7476196, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:52:55.205399 964 leader_election.cc:290] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5fa23b4772a944c0b0279eabd7476196 (127.0.105.62:44705), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
I20260501 13:52:55.229728 951 catalog_manager.cc:1380] Generated new certificate authority record
I20260501 13:52:55.231527 951 catalog_manager.cc:1514] Loading token signing keys...
I20260501 13:52:55.250952 968 mvcc.cc:204] Tried to move back new op lower bound from 7281228083518861312 to 7281228083097825280. Current Snapshot: MvccSnapshot[applied={T|T < 7281228083518861312}]
I20260501 13:52:55.256556 951 catalog_manager.cc:6044] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7: Generated new TSK 0
I20260501 13:52:55.257930 951 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260501 13:52:55.260284 951 catalog_manager.cc:1537] Loading latest processed Hive Metastore notification log event ID...
I20260501 13:52:55.261406 966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:52:55.261716 966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [sys.catalog]: This master's current role is: FOLLOWER
I20260501 13:52:55.262058 964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader e47102ddbd4f41c1bf0a2864d97237b7. Latest consensus state: current_term: 1 leader_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:52:55.262739 964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [sys.catalog]: This master's current role is: FOLLOWER
I20260501 13:52:55.262941 967 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260501 13:52:55.263294 951 catalog_manager.cc:5893] Last processed Hive Metastore notification event ID: -1
I20260501 13:52:55.264683 926 sys_catalog.cc:455] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:52:55.264899 926 sys_catalog.cc:458] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [sys.catalog]: This master's current role is: LEADER
I20260501 13:52:55.275204 420 external_mini_cluster.cc:1366] Running /tmp/dist-test-task4ezAfy/build/asan/bin/kudu
/tmp/dist-test-task4ezAfy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.105.1:0
--local_ip_for_outbound_sockets=127.0.105.1
--webserver_interface=127.0.105.1
--webserver_port=0
--tserver_master_addrs=127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729
--builtin_ntp_servers=127.0.105.20:38455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260501 13:52:55.352778 986 catalog_manager.cc:1269] Loaded cluster ID: 6424d308ce1d424b8fe7d2c1b812f4f7
I20260501 13:52:55.352892 986 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca: loading cluster ID for follower catalog manager: success
I20260501 13:52:55.356878 986 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca: acquiring CA information for follower catalog manager: success
I20260501 13:52:55.359916 986 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260501 13:52:55.772306 988 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e47102ddbd4f41c1bf0a2864d97237b7)
I20260501 13:52:55.772723 988 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:52:55.774925 815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5fa23b4772a944c0b0279eabd7476196" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84a39a5f167243ff9bfe07189aa881ca" is_pre_election: true
I20260501 13:52:55.776913 886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5fa23b4772a944c0b0279eabd7476196" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" is_pre_election: true
I20260501 13:52:55.778077 720 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:52:55.783631 988 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
I20260501 13:52:55.784243 989 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260501 13:52:55.827497 975 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260501 13:52:55.827847 975 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260501 13:52:55.827984 975 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260501 13:52:55.838716 975 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260501 13:52:55.839048 975 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.105.1
I20260501 13:52:55.856065 975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.105.20:38455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.105.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.105.1
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.105.1
--log_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision aa7cc0aa9c559b87f264235eaadd03ec492277e4
build type FASTDEBUG
built by None at 01 May 2026 13:43:23 UTC on bdcb31816ec0
build id 11694
ASAN enabled
I20260501 13:52:55.858290 975 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260501 13:52:55.860651 975 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260501 13:52:55.873189 994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260501 13:52:55.874315 995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260501 13:52:55.875912 997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260501 13:52:55.876971 975 server_base.cc:1061] running on GCE node
I20260501 13:52:55.877698 975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260501 13:52:55.879015 975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260501 13:52:55.880307 975 hybrid_clock.cc:648] HybridClock initialized: now 1777643575880188 us; error 110 us; skew 500 ppm
I20260501 13:52:55.880716 975 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260501 13:52:55.884243 975 webserver.cc:492] Webserver started at http://127.0.105.1:43311/ using document root <none> and password file <none>
I20260501 13:52:55.885251 975 fs_manager.cc:362] Metadata directory not provided
I20260501 13:52:55.885387 975 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260501 13:52:55.885821 975 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260501 13:52:55.889084 975 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-0/data/instance:
uuid: "e4a8a400d10343718a43d5fcc669dd80"
format_stamp: "Formatted at 2026-05-01 13:52:55 on dist-test-slave-qlwv"
I20260501 13:52:55.890084 975 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-0/wal/instance:
uuid: "e4a8a400d10343718a43d5fcc669dd80"
format_stamp: "Formatted at 2026-05-01 13:52:55 on dist-test-slave-qlwv"
I20260501 13:52:55.896991 975 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260501 13:52:55.901373 1003 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260501 13:52:55.903060 975 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260501 13:52:55.903276 975 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-0/data,/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-0/wal
uuid: "e4a8a400d10343718a43d5fcc669dd80"
format_stamp: "Formatted at 2026-05-01 13:52:55 on dist-test-slave-qlwv"
I20260501 13:52:55.903509 975 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260501 13:52:55.923619 975 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260501 13:52:55.924773 975 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260501 13:52:55.925052 975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260501 13:52:55.926213 975 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260501 13:52:55.928431 975 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260501 13:52:55.928540 975 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260501 13:52:55.928686 975 ts_tablet_manager.cc:616] Registered 0 tablets
I20260501 13:52:55.928761 975 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260501 13:52:55.933344 937 catalog_manager.cc:1269] Loaded cluster ID: 6424d308ce1d424b8fe7d2c1b812f4f7
I20260501 13:52:55.933439 937 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: loading cluster ID for follower catalog manager: success
I20260501 13:52:55.937319 937 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: acquiring CA information for follower catalog manager: success
I20260501 13:52:55.939775 937 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260501 13:52:55.984390 975 rpc_server.cc:307] RPC server started. Bound to: 127.0.105.1:44349
I20260501 13:52:55.984460 1115 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.105.1:44349 every 8 connection(s)
I20260501 13:52:55.985775 975 server_base.cc:1193] Dumped server information to /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-0/data/info.pb
I20260501 13:52:55.992228 420 external_mini_cluster.cc:1428] Started /tmp/dist-test-task4ezAfy/build/asan/bin/kudu as pid 975
I20260501 13:52:55.992486 420 external_mini_cluster.cc:1442] Reading /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-0/wal/instance
I20260501 13:52:56.000270 420 external_mini_cluster.cc:1366] Running /tmp/dist-test-task4ezAfy/build/asan/bin/kudu
/tmp/dist-test-task4ezAfy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.105.2:0
--local_ip_for_outbound_sockets=127.0.105.2
--webserver_interface=127.0.105.2
--webserver_port=0
--tserver_master_addrs=127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729
--builtin_ntp_servers=127.0.105.20:38455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260501 13:52:56.010070 1118 heartbeater.cc:344] Connected to a master server at 127.0.105.61:33209
I20260501 13:52:56.010752 1116 heartbeater.cc:344] Connected to a master server at 127.0.105.60:39729
I20260501 13:52:56.010943 1116 heartbeater.cc:461] Registering TS with master...
I20260501 13:52:56.010951 1118 heartbeater.cc:461] Registering TS with master...
I20260501 13:52:56.012192 1118 heartbeater.cc:507] Master 127.0.105.61:33209 requested a full tablet report, sending...
I20260501 13:52:56.012215 1116 heartbeater.cc:507] Master 127.0.105.60:39729 requested a full tablet report, sending...
I20260501 13:52:56.013903 1117 heartbeater.cc:344] Connected to a master server at 127.0.105.62:44705
I20260501 13:52:56.014051 1117 heartbeater.cc:461] Registering TS with master...
I20260501 13:52:56.014575 1117 heartbeater.cc:507] Master 127.0.105.62:44705 requested a full tablet report, sending...
I20260501 13:52:56.014581 876 ts_manager.cc:194] Registered new tserver with Master: e4a8a400d10343718a43d5fcc669dd80 (127.0.105.1:44349)
I20260501 13:52:56.014925 806 ts_manager.cc:194] Registered new tserver with Master: e4a8a400d10343718a43d5fcc669dd80 (127.0.105.1:44349)
I20260501 13:52:56.016180 735 ts_manager.cc:194] Registered new tserver with Master: e4a8a400d10343718a43d5fcc669dd80 (127.0.105.1:44349)
I20260501 13:52:56.016754 876 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.105.1:59869
I20260501 13:52:56.153018 1127 raft_consensus.cc:493] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e47102ddbd4f41c1bf0a2864d97237b7)
I20260501 13:52:56.153198 1127 raft_consensus.cc:515] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:52:56.154052 1127 leader_election.cc:290] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5fa23b4772a944c0b0279eabd7476196 (127.0.105.62:44705), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
I20260501 13:52:56.154693 746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "84a39a5f167243ff9bfe07189aa881ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5fa23b4772a944c0b0279eabd7476196" is_pre_election: true
I20260501 13:52:56.154817 886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "84a39a5f167243ff9bfe07189aa881ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" is_pre_election: true
I20260501 13:52:56.155448 790 leader_election.cc:304] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84a39a5f167243ff9bfe07189aa881ca; no voters: 5fa23b4772a944c0b0279eabd7476196, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:52:56.155732 1127 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260501 13:52:56.375025 1126 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260501 13:52:56.375461 1126 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260501 13:52:56.375613 1126 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260501 13:52:56.384812 1126 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260501 13:52:56.385257 1126 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.105.2
I20260501 13:52:56.397254 1126 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.105.20:38455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.105.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.105.2
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.105.2
--log_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision aa7cc0aa9c559b87f264235eaadd03ec492277e4
build type FASTDEBUG
built by None at 01 May 2026 13:43:23 UTC on bdcb31816ec0
build id 11694
ASAN enabled
I20260501 13:52:56.399194 1126 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260501 13:52:56.401907 1126 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260501 13:52:56.411557 1132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260501 13:52:56.412496 1133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260501 13:52:56.413400 1136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260501 13:52:56.413648 1126 server_base.cc:1061] running on GCE node
I20260501 13:52:56.414280 1126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260501 13:52:56.415155 1126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260501 13:52:56.416402 1126 hybrid_clock.cc:648] HybridClock initialized: now 1777643576416360 us; error 33 us; skew 500 ppm
I20260501 13:52:56.416706 1126 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260501 13:52:56.418776 1126 webserver.cc:492] Webserver started at http://127.0.105.2:33999/ using document root <none> and password file <none>
I20260501 13:52:56.420082 1126 fs_manager.cc:362] Metadata directory not provided
I20260501 13:52:56.420200 1126 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260501 13:52:56.420586 1126 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260501 13:52:56.423015 1126 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-1/data/instance:
uuid: "1151db12ecae4dd7bb88b3d1415b7d92"
format_stamp: "Formatted at 2026-05-01 13:52:56 on dist-test-slave-qlwv"
I20260501 13:52:56.423848 1126 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-1/wal/instance:
uuid: "1151db12ecae4dd7bb88b3d1415b7d92"
format_stamp: "Formatted at 2026-05-01 13:52:56 on dist-test-slave-qlwv"
I20260501 13:52:56.429126 1126 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260501 13:52:56.432561 1141 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260501 13:52:56.433884 1126 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260501 13:52:56.434053 1126 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-1/data,/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-1/wal
uuid: "1151db12ecae4dd7bb88b3d1415b7d92"
format_stamp: "Formatted at 2026-05-01 13:52:56 on dist-test-slave-qlwv"
I20260501 13:52:56.434250 1126 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260501 13:52:56.451117 1126 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260501 13:52:56.452472 1126 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260501 13:52:56.452749 1126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260501 13:52:56.453653 1126 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260501 13:52:56.455232 1126 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260501 13:52:56.455359 1126 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260501 13:52:56.455480 1126 ts_tablet_manager.cc:616] Registered 0 tablets
I20260501 13:52:56.455562 1126 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260501 13:52:56.492709 1126 rpc_server.cc:307] RPC server started. Bound to: 127.0.105.2:43565
I20260501 13:52:56.492798 1253 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.105.2:43565 every 8 connection(s)
I20260501 13:52:56.494238 1126 server_base.cc:1193] Dumped server information to /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-1/data/info.pb
I20260501 13:52:56.495712 420 external_mini_cluster.cc:1428] Started /tmp/dist-test-task4ezAfy/build/asan/bin/kudu as pid 1126
I20260501 13:52:56.495941 420 external_mini_cluster.cc:1442] Reading /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-1/wal/instance
I20260501 13:52:56.500955 420 external_mini_cluster.cc:1366] Running /tmp/dist-test-task4ezAfy/build/asan/bin/kudu
/tmp/dist-test-task4ezAfy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.105.3:0
--local_ip_for_outbound_sockets=127.0.105.3
--webserver_interface=127.0.105.3
--webserver_port=0
--tserver_master_addrs=127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729
--builtin_ntp_servers=127.0.105.20:38455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260501 13:52:56.514503 1255 heartbeater.cc:344] Connected to a master server at 127.0.105.62:44705
I20260501 13:52:56.514842 1255 heartbeater.cc:461] Registering TS with master...
I20260501 13:52:56.515348 1256 heartbeater.cc:344] Connected to a master server at 127.0.105.61:33209
I20260501 13:52:56.515589 1256 heartbeater.cc:461] Registering TS with master...
I20260501 13:52:56.515911 1255 heartbeater.cc:507] Master 127.0.105.62:44705 requested a full tablet report, sending...
I20260501 13:52:56.516201 1256 heartbeater.cc:507] Master 127.0.105.61:33209 requested a full tablet report, sending...
I20260501 13:52:56.517091 1254 heartbeater.cc:344] Connected to a master server at 127.0.105.60:39729
I20260501 13:52:56.517210 1254 heartbeater.cc:461] Registering TS with master...
I20260501 13:52:56.517707 1254 heartbeater.cc:507] Master 127.0.105.60:39729 requested a full tablet report, sending...
I20260501 13:52:56.517741 806 ts_manager.cc:194] Registered new tserver with Master: 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
I20260501 13:52:56.518631 876 ts_manager.cc:194] Registered new tserver with Master: 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
I20260501 13:52:56.519469 876 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.105.2:48403
I20260501 13:52:56.519474 1116 heartbeater.cc:499] Master 127.0.105.60:39729 was elected leader, sending a full tablet report...
I20260501 13:52:56.519546 735 ts_manager.cc:194] Registered new tserver with Master: 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:52:56.874584 1261 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260501 13:52:56.874929 1261 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260501 13:52:56.875064 1261 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260501 13:52:56.883790 1261 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260501 13:52:56.884004 1261 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.105.3
I20260501 13:52:56.894620 1261 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.105.20:38455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.105.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.105.3
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.105.3
--log_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision aa7cc0aa9c559b87f264235eaadd03ec492277e4
build type FASTDEBUG
built by None at 01 May 2026 13:43:23 UTC on bdcb31816ec0
build id 11694
ASAN enabled
I20260501 13:52:56.896306 1261 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260501 13:52:56.898089 1261 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260501 13:52:56.908532 1266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260501 13:52:56.908780 1267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260501 13:52:56.909484 1269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260501 13:52:56.910328 1261 server_base.cc:1061] running on GCE node
I20260501 13:52:56.910938 1261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260501 13:52:56.912052 1261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260501 13:52:56.913499 1261 hybrid_clock.cc:648] HybridClock initialized: now 1777643576913436 us; error 51 us; skew 500 ppm
I20260501 13:52:56.913851 1261 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260501 13:52:56.916036 1261 webserver.cc:492] Webserver started at http://127.0.105.3:33459/ using document root <none> and password file <none>
I20260501 13:52:56.916807 1261 fs_manager.cc:362] Metadata directory not provided
I20260501 13:52:56.916937 1261 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260501 13:52:56.917289 1261 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260501 13:52:56.919896 1261 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-2/data/instance:
uuid: "b0226350eae044398f04ef4e5cbb550e"
format_stamp: "Formatted at 2026-05-01 13:52:56 on dist-test-slave-qlwv"
I20260501 13:52:56.920717 1261 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-2/wal/instance:
uuid: "b0226350eae044398f04ef4e5cbb550e"
format_stamp: "Formatted at 2026-05-01 13:52:56 on dist-test-slave-qlwv"
I20260501 13:52:56.926471 1261 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260501 13:52:56.930014 1275 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260501 13:52:56.931597 1261 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260501 13:52:56.931813 1261 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-2/data,/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-2/wal
uuid: "b0226350eae044398f04ef4e5cbb550e"
format_stamp: "Formatted at 2026-05-01 13:52:56 on dist-test-slave-qlwv"
I20260501 13:52:56.932025 1261 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260501 13:52:56.964941 1261 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260501 13:52:56.965821 1261 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260501 13:52:56.966107 1261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260501 13:52:56.967054 1261 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260501 13:52:56.968720 1261 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260501 13:52:56.968837 1261 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260501 13:52:56.968966 1261 ts_tablet_manager.cc:616] Registered 0 tablets
I20260501 13:52:56.969048 1261 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260501 13:52:57.008911 1261 rpc_server.cc:307] RPC server started. Bound to: 127.0.105.3:41841
I20260501 13:52:57.008961 1387 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.105.3:41841 every 8 connection(s)
I20260501 13:52:57.010488 1261 server_base.cc:1193] Dumped server information to /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-2/data/info.pb
I20260501 13:52:57.014034 420 external_mini_cluster.cc:1428] Started /tmp/dist-test-task4ezAfy/build/asan/bin/kudu as pid 1261
I20260501 13:52:57.014252 420 external_mini_cluster.cc:1442] Reading /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-2/wal/instance
I20260501 13:52:57.022207 1254 heartbeater.cc:499] Master 127.0.105.60:39729 was elected leader, sending a full tablet report...
I20260501 13:52:57.034228 1389 heartbeater.cc:344] Connected to a master server at 127.0.105.62:44705
I20260501 13:52:57.034839 1389 heartbeater.cc:461] Registering TS with master...
I20260501 13:52:57.036149 1389 heartbeater.cc:507] Master 127.0.105.62:44705 requested a full tablet report, sending...
I20260501 13:52:57.038313 735 ts_manager.cc:194] Registered new tserver with Master: b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841)
I20260501 13:52:57.043531 1388 heartbeater.cc:344] Connected to a master server at 127.0.105.60:39729
I20260501 13:52:57.043704 1388 heartbeater.cc:461] Registering TS with master...
I20260501 13:52:57.044211 1388 heartbeater.cc:507] Master 127.0.105.60:39729 requested a full tablet report, sending...
I20260501 13:52:57.044576 1390 heartbeater.cc:344] Connected to a master server at 127.0.105.61:33209
I20260501 13:52:57.044721 1390 heartbeater.cc:461] Registering TS with master...
I20260501 13:52:57.045207 1390 heartbeater.cc:507] Master 127.0.105.61:33209 requested a full tablet report, sending...
I20260501 13:52:57.045300 876 ts_manager.cc:194] Registered new tserver with Master: b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841)
I20260501 13:52:57.046381 876 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.105.3:40075
I20260501 13:52:57.047183 805 ts_manager.cc:194] Registered new tserver with Master: b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841)
I20260501 13:52:57.060818 420 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20260501 13:52:57.096051 876 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35630:
name: "default.Table_86cdd461d6044be8b0ae522dfcc0714a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260501 13:52:57.096357 874 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35630:
name: "default.Table_06eba6fa1b4949ffa139c461e70349f9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260501 13:52:57.097162 873 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35630:
name: "default.Table_2ff058e2a2324419b03d708582f033d5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260501 13:52:57.097872 872 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35630:
name: "default.Table_d7ed2811199d451c80270ccf0c6c80f4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260501 13:52:57.098994 871 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35630:
name: "default.Table_293c891afe6c4d609f8069858909d21b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260501 13:52:57.099444 869 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35630:
name: "default.Table_5a6d83f7292440209933a379074aadbb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260501 13:52:57.100024 868 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35630:
name: "default.Table_8beb55fb2bdd42309d0183432704b989"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260501 13:52:57.100078 867 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35630:
name: "default.Table_5a0a455cca3c4ab6a059aac962b7cea0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260501 13:52:57.100450 870 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35630:
name: "default.Table_b93ea2b7623547349398aaf4f3d85376"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260501 13:52:57.096076 875 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35630:
name: "default.Table_49594ae63df046b38ccad889763a2723"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260501 13:52:57.107616 876 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_86cdd461d6044be8b0ae522dfcc0714a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260501 13:52:57.109159 420 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-task4ezAfy/build/asan/bin/kudu pid 842
I20260501 13:52:57.112779 906 generic_service.cc:196] Checking for leaks (request via RPC)
I20260501 13:52:57.235796 1439 raft_consensus.cc:493] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e47102ddbd4f41c1bf0a2864d97237b7)
I20260501 13:52:57.236239 1439 raft_consensus.cc:515] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:52:57.237571 746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "84a39a5f167243ff9bfe07189aa881ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5fa23b4772a944c0b0279eabd7476196" is_pre_election: true
I20260501 13:52:57.244014 1439 leader_election.cc:290] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5fa23b4772a944c0b0279eabd7476196 (127.0.105.62:44705), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
I20260501 13:52:57.284160 1440 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e47102ddbd4f41c1bf0a2864d97237b7)
I20260501 13:52:57.284577 1440 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:52:57.286021 815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5fa23b4772a944c0b0279eabd7476196" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84a39a5f167243ff9bfe07189aa881ca" is_pre_election: true
I20260501 13:52:57.286278 815 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fa23b4772a944c0b0279eabd7476196 in term 1.
I20260501 13:52:57.286861 723 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196, 84a39a5f167243ff9bfe07189aa881ca; no voters:
I20260501 13:52:57.287003 1440 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
I20260501 13:52:57.287187 1440 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260501 13:52:57.287356 1440 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e47102ddbd4f41c1bf0a2864d97237b7)
I20260501 13:52:57.287526 1440 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 1 FOLLOWER]: Advancing to term 2
I20260501 13:52:57.290867 1440 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:52:57.291841 1440 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 2 election: Requested vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
I20260501 13:52:57.297441 815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5fa23b4772a944c0b0279eabd7476196" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84a39a5f167243ff9bfe07189aa881ca"
I20260501 13:52:57.297689 815 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 1 FOLLOWER]: Advancing to term 2
I20260501 13:52:57.300912 815 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fa23b4772a944c0b0279eabd7476196 in term 2.
I20260501 13:52:57.301640 723 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196, 84a39a5f167243ff9bfe07189aa881ca; no voters:
I20260501 13:52:57.302433 886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5fa23b4772a944c0b0279eabd7476196" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" is_pre_election: true
I20260501 13:52:57.303478 885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5fa23b4772a944c0b0279eabd7476196" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e47102ddbd4f41c1bf0a2864d97237b7"
I20260501 13:52:57.303613 815 raft_consensus.cc:1240] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 2 FOLLOWER]: Rejecting Update request from peer e47102ddbd4f41c1bf0a2864d97237b7 for earlier term 1. Current term is 2. Ops: []
I20260501 13:52:57.302927 1441 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 2 FOLLOWER]: Leader election won for term 2
I20260501 13:52:57.303980 1441 raft_consensus.cc:697] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 2 LEADER]: Becoming Leader. State: Replica: 5fa23b4772a944c0b0279eabd7476196, State: Running, Role: LEADER
I20260501 13:52:57.304185 420 external_mini_cluster.cc:1658] Killing /tmp/dist-test-task4ezAfy/build/asan/bin/kudu with pid 842
I20260501 13:52:57.304137 945 consensus_queue.cc:1059] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260501 13:52:57.305171 1441 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:52:57.333838 1440 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5fa23b4772a944c0b0279eabd7476196. Latest consensus state: current_term: 2 leader_uuid: "5fa23b4772a944c0b0279eabd7476196" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:52:57.334113 1440 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: This master's current role is: LEADER
W20260501 13:52:57.338135 1398 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.105.60:39729)
W20260501 13:52:57.355192 1403 negotiation.cc:336] Failed RPC negotiation. Trace:
0501 13:52:57.342789 (+ 0us) reactor.cc:678] Submitting negotiation task for client connection to 127.0.105.60:39729 (local address 127.0.0.1:35642)
0501 13:52:57.343138 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0501 13:52:57.343160 (+ 22us) client_negotiation.cc:175] Beginning negotiation
0501 13:52:57.343404 (+ 244us) client_negotiation.cc:253] Sending NEGOTIATE NegotiatePB request
0501 13:52:57.353657 (+ 10253us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":99}
W20260501 13:52:57.365712 1446 negotiation.cc:336] Failed RPC negotiation. Trace:
0501 13:52:57.353408 (+ 0us) reactor.cc:678] Submitting negotiation task for client connection to 127.0.105.60:39729 (local address 127.0.0.1:35652)
0501 13:52:57.364312 (+ 10904us) negotiation.cc:107] Waiting for socket to connect
0501 13:52:57.364517 (+ 205us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":10772,"threads_started":1}
W20260501 13:52:57.369376 720 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection reset by peer (error 104)
I20260501 13:52:57.366722 1443 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260501 13:52:57.372565 1443 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260501 13:52:57.373759 1443 catalog_manager.cc:1269] Loaded cluster ID: 6424d308ce1d424b8fe7d2c1b812f4f7
I20260501 13:52:57.373912 1443 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260501 13:52:57.375490 1443 catalog_manager.cc:1514] Loading token signing keys...
I20260501 13:52:57.396253 1443 catalog_manager.cc:6055] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: Loaded TSK: 0
I20260501 13:52:57.400168 1443 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260501 13:52:57.401492 1443 catalog_manager.cc:1537] Loading latest processed Hive Metastore notification log event ID...
W20260501 13:52:57.406111 790 leader_election.cc:336] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:52:57.406456 790 leader_election.cc:304] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84a39a5f167243ff9bfe07189aa881ca; no voters: 5fa23b4772a944c0b0279eabd7476196, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:52:57.406812 1439 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260501 13:52:57.408041 1443 catalog_manager.cc:5893] Last processed Hive Metastore notification event ID: -1
I20260501 13:52:57.421077 734 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35974:
name: "default.Table_5a6d83f7292440209933a379074aadbb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260501 13:52:57.422233 735 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35974:
name: "default.Table_5a0a455cca3c4ab6a059aac962b7cea0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260501 13:52:57.426714 736 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35974:
name: "default.Table_8beb55fb2bdd42309d0183432704b989"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260501 13:52:57.427300 727 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35974:
name: "default.Table_293c891afe6c4d609f8069858909d21b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260501 13:52:57.428220 730 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35974:
name: "default.Table_06eba6fa1b4949ffa139c461e70349f9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260501 13:52:57.428745 728 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35974:
name: "default.Table_d7ed2811199d451c80270ccf0c6c80f4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260501 13:52:57.423247 733 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35974:
name: "default.Table_b93ea2b7623547349398aaf4f3d85376"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260501 13:52:57.421610 729 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35974:
name: "default.Table_2ff058e2a2324419b03d708582f033d5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260501 13:52:57.429534 732 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35974:
name: "default.Table_49594ae63df046b38ccad889763a2723"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260501 13:52:57.429792 731 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35974:
name: "default.Table_86cdd461d6044be8b0ae522dfcc0714a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260501 13:52:57.435406 735 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_5a0a455cca3c4ab6a059aac962b7cea0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260501 13:52:57.538657 1116 heartbeater.cc:646] Failed to heartbeat to 127.0.105.60:39729 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:52:57.549000 1388 heartbeater.cc:499] Master 127.0.105.60:39729 was elected leader, sending a full tablet report...
W20260501 13:52:57.556491 1388 heartbeater.cc:646] Failed to heartbeat to 127.0.105.60:39729 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
W20260501 13:52:57.569018 1254 heartbeater.cc:646] Failed to heartbeat to 127.0.105.60:39729 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:52:57.668123 735 catalog_manager.cc:2174] created HMS catalog entry for table [id=8818494b487941a4877edf5b079d9756]
W20260501 13:52:57.676318 736 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_8beb55fb2bdd42309d0183432704b989 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260501 13:52:57.676580 727 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_293c891afe6c4d609f8069858909d21b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260501 13:52:57.677604 730 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_06eba6fa1b4949ffa139c461e70349f9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260501 13:52:57.678561 728 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_d7ed2811199d451c80270ccf0c6c80f4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260501 13:52:57.679468 733 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_b93ea2b7623547349398aaf4f3d85376 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260501 13:52:57.680331 729 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_2ff058e2a2324419b03d708582f033d5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260501 13:52:57.681285 732 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_49594ae63df046b38ccad889763a2723 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260501 13:52:57.682210 731 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_86cdd461d6044be8b0ae522dfcc0714a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260501 13:52:57.682883 734 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_5a6d83f7292440209933a379074aadbb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260501 13:52:57.698149 815 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 2 FOLLOWER]: Refusing update from remote peer 5fa23b4772a944c0b0279eabd7476196: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
W20260501 13:52:57.698541 720 consensus_peers.cc:597] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 -> Peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Couldn't send request to peer e47102ddbd4f41c1bf0a2864d97237b7. Status: Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260501 13:52:57.698886 1441 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260501 13:52:57.719931 1439 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5fa23b4772a944c0b0279eabd7476196. Latest consensus state: current_term: 2 leader_uuid: "5fa23b4772a944c0b0279eabd7476196" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:52:57.720191 1439 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [sys.catalog]: This master's current role is: FOLLOWER
I20260501 13:52:57.731477 1440 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "5fa23b4772a944c0b0279eabd7476196" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:52:57.731752 1440 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: This master's current role is: LEADER
I20260501 13:52:57.732893 1439 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "5fa23b4772a944c0b0279eabd7476196" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:52:57.733085 1439 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [sys.catalog]: This master's current role is: FOLLOWER
I20260501 13:52:57.739303 735 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35974:
table { table_name: "default.Table_5a0a455cca3c4ab6a059aac962b7cea0" } new_table_name: "default.Table_38e9fe72f57e4e9e927e2fd11e3dc180" modify_external_catalogs: true
I20260501 13:52:57.744055 736 catalog_manager.cc:2174] created HMS catalog entry for table [id=f7226f45b9d94af2ad176c2d4a419e88]
I20260501 13:52:57.772966 1189 tablet_service.cc:1511] Processing CreateTablet for tablet c6507fffd6a4466ab15b1eb89c7f32c7 (DEFAULT_TABLE table=default.table_5a0a455cca3c4ab6a059aac962b7cea0 [id=8818494b487941a4877edf5b079d9756]), partition=RANGE (key) PARTITION UNBOUNDED
I20260501 13:52:57.774770 1189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6507fffd6a4466ab15b1eb89c7f32c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260501 13:52:57.787086 736 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35974:
table { table_name: "default.Table_8beb55fb2bdd42309d0183432704b989" } new_table_name: "default.Table_62e7535549cf414f8af163f82d73b382" modify_external_catalogs: true
I20260501 13:52:57.801553 1051 tablet_service.cc:1511] Processing CreateTablet for tablet c6507fffd6a4466ab15b1eb89c7f32c7 (DEFAULT_TABLE table=default.table_5a0a455cca3c4ab6a059aac962b7cea0 [id=8818494b487941a4877edf5b079d9756]), partition=RANGE (key) PARTITION UNBOUNDED
I20260501 13:52:57.803540 1323 tablet_service.cc:1511] Processing CreateTablet for tablet c6507fffd6a4466ab15b1eb89c7f32c7 (DEFAULT_TABLE table=default.table_5a0a455cca3c4ab6a059aac962b7cea0 [id=8818494b487941a4877edf5b079d9756]), partition=RANGE (key) PARTITION UNBOUNDED
I20260501 13:52:57.805444 1050 tablet_service.cc:1511] Processing CreateTablet for tablet 546264ae15814277b0acfb77598562df (DEFAULT_TABLE table=default.table_8beb55fb2bdd42309d0183432704b989 [id=f7226f45b9d94af2ad176c2d4a419e88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260501 13:52:57.805500 1188 tablet_service.cc:1511] Processing CreateTablet for tablet 546264ae15814277b0acfb77598562df (DEFAULT_TABLE table=default.table_8beb55fb2bdd42309d0183432704b989 [id=f7226f45b9d94af2ad176c2d4a419e88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260501 13:52:57.806041 1188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 546264ae15814277b0acfb77598562df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260501 13:52:57.806051 1051 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6507fffd6a4466ab15b1eb89c7f32c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260501 13:52:57.806476 1050 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 546264ae15814277b0acfb77598562df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260501 13:52:57.807463 1323 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6507fffd6a4466ab15b1eb89c7f32c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260501 13:52:57.807780 1322 tablet_service.cc:1511] Processing CreateTablet for tablet 546264ae15814277b0acfb77598562df (DEFAULT_TABLE table=default.table_8beb55fb2bdd42309d0183432704b989 [id=f7226f45b9d94af2ad176c2d4a419e88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260501 13:52:57.808336 1322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 546264ae15814277b0acfb77598562df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260501 13:52:57.825774 1465 tablet_bootstrap.cc:492] T 546264ae15814277b0acfb77598562df P e4a8a400d10343718a43d5fcc669dd80: Bootstrap starting.
I20260501 13:52:57.829658 1465 tablet_bootstrap.cc:654] T 546264ae15814277b0acfb77598562df P e4a8a400d10343718a43d5fcc669dd80: Neither blocks nor log segments found. Creating new log.
I20260501 13:52:57.831202 1465 log.cc:826] T 546264ae15814277b0acfb77598562df P e4a8a400d10343718a43d5fcc669dd80: Log is configured to *not* fsync() on all Append() calls
I20260501 13:52:57.833127 1464 tablet_bootstrap.cc:492] T 546264ae15814277b0acfb77598562df P b0226350eae044398f04ef4e5cbb550e: Bootstrap starting.
I20260501 13:52:57.838151 1464 tablet_bootstrap.cc:654] T 546264ae15814277b0acfb77598562df P b0226350eae044398f04ef4e5cbb550e: Neither blocks nor log segments found. Creating new log.
I20260501 13:52:57.839430 1463 tablet_bootstrap.cc:492] T 546264ae15814277b0acfb77598562df P 1151db12ecae4dd7bb88b3d1415b7d92: Bootstrap starting.
I20260501 13:52:57.839805 1464 log.cc:826] T 546264ae15814277b0acfb77598562df P b0226350eae044398f04ef4e5cbb550e: Log is configured to *not* fsync() on all Append() calls
I20260501 13:52:57.842167 1465 tablet_bootstrap.cc:492] T 546264ae15814277b0acfb77598562df P e4a8a400d10343718a43d5fcc669dd80: No bootstrap required, opened a new log
I20260501 13:52:57.842546 1465 ts_tablet_manager.cc:1403] T 546264ae15814277b0acfb77598562df P e4a8a400d10343718a43d5fcc669dd80: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.004s
I20260501 13:52:57.848939 1464 tablet_bootstrap.cc:492] T 546264ae15814277b0acfb77598562df P b0226350eae044398f04ef4e5cbb550e: No bootstrap required, opened a new log
I20260501 13:52:57.849339 1464 ts_tablet_manager.cc:1403] T 546264ae15814277b0acfb77598562df P b0226350eae044398f04ef4e5cbb550e: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.009s
I20260501 13:52:57.851084 1463 tablet_bootstrap.cc:654] T 546264ae15814277b0acfb77598562df P 1151db12ecae4dd7bb88b3d1415b7d92: Neither blocks nor log segments found. Creating new log.
I20260501 13:52:57.854956 1465 raft_consensus.cc:359] T 546264ae15814277b0acfb77598562df P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:57.856930 1463 log.cc:826] T 546264ae15814277b0acfb77598562df P 1151db12ecae4dd7bb88b3d1415b7d92: Log is configured to *not* fsync() on all Append() calls
I20260501 13:52:57.856663 1464 raft_consensus.cc:359] T 546264ae15814277b0acfb77598562df P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:57.857259 1464 raft_consensus.cc:385] T 546264ae15814277b0acfb77598562df P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260501 13:52:57.857401 1464 raft_consensus.cc:740] T 546264ae15814277b0acfb77598562df P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0226350eae044398f04ef4e5cbb550e, State: Initialized, Role: FOLLOWER
I20260501 13:52:57.857592 1465 raft_consensus.cc:385] T 546264ae15814277b0acfb77598562df P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260501 13:52:57.857769 1465 raft_consensus.cc:740] T 546264ae15814277b0acfb77598562df P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4a8a400d10343718a43d5fcc669dd80, State: Initialized, Role: FOLLOWER
I20260501 13:52:57.858310 1464 consensus_queue.cc:260] T 546264ae15814277b0acfb77598562df P b0226350eae044398f04ef4e5cbb550e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:57.858424 1465 consensus_queue.cc:260] T 546264ae15814277b0acfb77598562df P e4a8a400d10343718a43d5fcc669dd80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:57.863631 1463 tablet_bootstrap.cc:492] T 546264ae15814277b0acfb77598562df P 1151db12ecae4dd7bb88b3d1415b7d92: No bootstrap required, opened a new log
I20260501 13:52:57.864228 1389 heartbeater.cc:499] Master 127.0.105.62:44705 was elected leader, sending a full tablet report...
I20260501 13:52:57.864339 1464 ts_tablet_manager.cc:1434] T 546264ae15814277b0acfb77598562df P b0226350eae044398f04ef4e5cbb550e: Time spent starting tablet: real 0.015s user 0.005s sys 0.007s
I20260501 13:52:57.864606 1463 ts_tablet_manager.cc:1403] T 546264ae15814277b0acfb77598562df P 1151db12ecae4dd7bb88b3d1415b7d92: Time spent bootstrapping tablet: real 0.026s user 0.010s sys 0.000s
I20260501 13:52:57.865343 1464 tablet_bootstrap.cc:492] T c6507fffd6a4466ab15b1eb89c7f32c7 P b0226350eae044398f04ef4e5cbb550e: Bootstrap starting.
I20260501 13:52:57.867322 1464 tablet_bootstrap.cc:654] T c6507fffd6a4466ab15b1eb89c7f32c7 P b0226350eae044398f04ef4e5cbb550e: Neither blocks nor log segments found. Creating new log.
I20260501 13:52:57.871579 1117 heartbeater.cc:499] Master 127.0.105.62:44705 was elected leader, sending a full tablet report...
I20260501 13:52:57.874572 1465 ts_tablet_manager.cc:1434] T 546264ae15814277b0acfb77598562df P e4a8a400d10343718a43d5fcc669dd80: Time spent starting tablet: real 0.032s user 0.013s sys 0.001s
I20260501 13:52:57.875782 1465 tablet_bootstrap.cc:492] T c6507fffd6a4466ab15b1eb89c7f32c7 P e4a8a400d10343718a43d5fcc669dd80: Bootstrap starting.
I20260501 13:52:57.877641 1464 tablet_bootstrap.cc:492] T c6507fffd6a4466ab15b1eb89c7f32c7 P b0226350eae044398f04ef4e5cbb550e: No bootstrap required, opened a new log
I20260501 13:52:57.877812 1464 ts_tablet_manager.cc:1403] T c6507fffd6a4466ab15b1eb89c7f32c7 P b0226350eae044398f04ef4e5cbb550e: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.006s
I20260501 13:52:57.877941 1465 tablet_bootstrap.cc:654] T c6507fffd6a4466ab15b1eb89c7f32c7 P e4a8a400d10343718a43d5fcc669dd80: Neither blocks nor log segments found. Creating new log.
I20260501 13:52:57.878871 1464 raft_consensus.cc:359] T c6507fffd6a4466ab15b1eb89c7f32c7 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } }
I20260501 13:52:57.879068 1464 raft_consensus.cc:385] T c6507fffd6a4466ab15b1eb89c7f32c7 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260501 13:52:57.879160 1464 raft_consensus.cc:740] T c6507fffd6a4466ab15b1eb89c7f32c7 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0226350eae044398f04ef4e5cbb550e, State: Initialized, Role: FOLLOWER
I20260501 13:52:57.879494 1464 consensus_queue.cc:260] T c6507fffd6a4466ab15b1eb89c7f32c7 P b0226350eae044398f04ef4e5cbb550e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } }
I20260501 13:52:57.879998 1464 ts_tablet_manager.cc:1434] T c6507fffd6a4466ab15b1eb89c7f32c7 P b0226350eae044398f04ef4e5cbb550e: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260501 13:52:57.882762 1465 tablet_bootstrap.cc:492] T c6507fffd6a4466ab15b1eb89c7f32c7 P e4a8a400d10343718a43d5fcc669dd80: No bootstrap required, opened a new log
I20260501 13:52:57.882961 1465 ts_tablet_manager.cc:1403] T c6507fffd6a4466ab15b1eb89c7f32c7 P e4a8a400d10343718a43d5fcc669dd80: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.001s
I20260501 13:52:57.884035 1465 raft_consensus.cc:359] T c6507fffd6a4466ab15b1eb89c7f32c7 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } }
I20260501 13:52:57.884289 1465 raft_consensus.cc:385] T c6507fffd6a4466ab15b1eb89c7f32c7 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260501 13:52:57.884397 1465 raft_consensus.cc:740] T c6507fffd6a4466ab15b1eb89c7f32c7 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4a8a400d10343718a43d5fcc669dd80, State: Initialized, Role: FOLLOWER
I20260501 13:52:57.884792 1465 consensus_queue.cc:260] T c6507fffd6a4466ab15b1eb89c7f32c7 P e4a8a400d10343718a43d5fcc669dd80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } }
I20260501 13:52:57.885332 1465 ts_tablet_manager.cc:1434] T c6507fffd6a4466ab15b1eb89c7f32c7 P e4a8a400d10343718a43d5fcc669dd80: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260501 13:52:57.885197 1463 raft_consensus.cc:359] T 546264ae15814277b0acfb77598562df P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:57.886055 1463 raft_consensus.cc:385] T 546264ae15814277b0acfb77598562df P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260501 13:52:57.886176 1463 raft_consensus.cc:740] T 546264ae15814277b0acfb77598562df P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1151db12ecae4dd7bb88b3d1415b7d92, State: Initialized, Role: FOLLOWER
I20260501 13:52:57.886898 1463 consensus_queue.cc:260] T 546264ae15814277b0acfb77598562df P 1151db12ecae4dd7bb88b3d1415b7d92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:57.888031 1255 heartbeater.cc:499] Master 127.0.105.62:44705 was elected leader, sending a full tablet report...
I20260501 13:52:57.889806 1463 ts_tablet_manager.cc:1434] T 546264ae15814277b0acfb77598562df P 1151db12ecae4dd7bb88b3d1415b7d92: Time spent starting tablet: real 0.025s user 0.009s sys 0.002s
I20260501 13:52:57.890343 1463 tablet_bootstrap.cc:492] T c6507fffd6a4466ab15b1eb89c7f32c7 P 1151db12ecae4dd7bb88b3d1415b7d92: Bootstrap starting.
I20260501 13:52:57.892072 1463 tablet_bootstrap.cc:654] T c6507fffd6a4466ab15b1eb89c7f32c7 P 1151db12ecae4dd7bb88b3d1415b7d92: Neither blocks nor log segments found. Creating new log.
I20260501 13:52:57.893633 1463 tablet_bootstrap.cc:492] T c6507fffd6a4466ab15b1eb89c7f32c7 P 1151db12ecae4dd7bb88b3d1415b7d92: No bootstrap required, opened a new log
I20260501 13:52:57.893779 1463 ts_tablet_manager.cc:1403] T c6507fffd6a4466ab15b1eb89c7f32c7 P 1151db12ecae4dd7bb88b3d1415b7d92: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260501 13:52:57.894474 1463 raft_consensus.cc:359] T c6507fffd6a4466ab15b1eb89c7f32c7 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } }
I20260501 13:52:57.894637 1463 raft_consensus.cc:385] T c6507fffd6a4466ab15b1eb89c7f32c7 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260501 13:52:57.894742 1463 raft_consensus.cc:740] T c6507fffd6a4466ab15b1eb89c7f32c7 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1151db12ecae4dd7bb88b3d1415b7d92, State: Initialized, Role: FOLLOWER
I20260501 13:52:57.895012 1463 consensus_queue.cc:260] T c6507fffd6a4466ab15b1eb89c7f32c7 P 1151db12ecae4dd7bb88b3d1415b7d92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } }
I20260501 13:52:57.895507 1463 ts_tablet_manager.cc:1434] T c6507fffd6a4466ab15b1eb89c7f32c7 P 1151db12ecae4dd7bb88b3d1415b7d92: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260501 13:52:57.915730 727 catalog_manager.cc:2174] created HMS catalog entry for table [id=6d282d8a3f814452977f46b6054268e2]
I20260501 13:52:57.919445 420 external_mini_cluster.cc:1366] Running /tmp/dist-test-task4ezAfy/build/asan/bin/kudu
/tmp/dist-test-task4ezAfy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.60:39729
--webserver_interface=127.0.105.60
--webserver_port=43413
--builtin_ntp_servers=127.0.105.20:38455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729
--hive_metastore_uris=thrift://127.0.0.1:33681
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260501 13:52:57.930418 1470 raft_consensus.cc:493] T c6507fffd6a4466ab15b1eb89c7f32c7 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:52:57.930696 1470 raft_consensus.cc:515] T c6507fffd6a4466ab15b1eb89c7f32c7 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } }
I20260501 13:52:57.932904 1470 leader_election.cc:290] T c6507fffd6a4466ab15b1eb89c7f32c7 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
I20260501 13:52:57.943861 1209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6507fffd6a4466ab15b1eb89c7f32c7" candidate_uuid: "e4a8a400d10343718a43d5fcc669dd80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" is_pre_election: true
I20260501 13:52:57.944413 1209 raft_consensus.cc:2468] T c6507fffd6a4466ab15b1eb89c7f32c7 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4a8a400d10343718a43d5fcc669dd80 in term 0.
I20260501 13:52:57.945396 1007 leader_election.cc:304] T c6507fffd6a4466ab15b1eb89c7f32c7 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1151db12ecae4dd7bb88b3d1415b7d92, e4a8a400d10343718a43d5fcc669dd80; no voters:
I20260501 13:52:57.945983 1470 raft_consensus.cc:2804] T c6507fffd6a4466ab15b1eb89c7f32c7 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260501 13:52:57.946127 1470 raft_consensus.cc:493] T c6507fffd6a4466ab15b1eb89c7f32c7 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260501 13:52:57.946223 1470 raft_consensus.cc:3060] T c6507fffd6a4466ab15b1eb89c7f32c7 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Advancing to term 1
I20260501 13:52:57.946409 1322 tablet_service.cc:1511] Processing CreateTablet for tablet 5c413244650e4784981922b6502f6b97 (DEFAULT_TABLE table=default.table_293c891afe6c4d609f8069858909d21b [id=6d282d8a3f814452977f46b6054268e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260501 13:52:57.946893 1322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c413244650e4784981922b6502f6b97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260501 13:52:57.952312 1051 tablet_service.cc:1511] Processing CreateTablet for tablet 5c413244650e4784981922b6502f6b97 (DEFAULT_TABLE table=default.table_293c891afe6c4d609f8069858909d21b [id=6d282d8a3f814452977f46b6054268e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260501 13:52:57.952375 1188 tablet_service.cc:1511] Processing CreateTablet for tablet 5c413244650e4784981922b6502f6b97 (DEFAULT_TABLE table=default.table_293c891afe6c4d609f8069858909d21b [id=6d282d8a3f814452977f46b6054268e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260501 13:52:57.952813 1051 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c413244650e4784981922b6502f6b97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260501 13:52:57.958240 1188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c413244650e4784981922b6502f6b97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260501 13:52:57.972373 1464 tablet_bootstrap.cc:492] T 5c413244650e4784981922b6502f6b97 P b0226350eae044398f04ef4e5cbb550e: Bootstrap starting.
I20260501 13:52:57.974177 1464 tablet_bootstrap.cc:654] T 5c413244650e4784981922b6502f6b97 P b0226350eae044398f04ef4e5cbb550e: Neither blocks nor log segments found. Creating new log.
I20260501 13:52:57.978142 1463 tablet_bootstrap.cc:492] T 5c413244650e4784981922b6502f6b97 P 1151db12ecae4dd7bb88b3d1415b7d92: Bootstrap starting.
I20260501 13:52:57.979871 1463 tablet_bootstrap.cc:654] T 5c413244650e4784981922b6502f6b97 P 1151db12ecae4dd7bb88b3d1415b7d92: Neither blocks nor log segments found. Creating new log.
I20260501 13:52:57.982784 727 master_service.cc:954] ReplaceTablet: received request to replace tablet 546264ae15814277b0acfb77598562df from {username='slave'} at 127.0.0.1:35974
I20260501 13:52:57.984200 1470 raft_consensus.cc:515] T c6507fffd6a4466ab15b1eb89c7f32c7 P e4a8a400d10343718a43d5fcc669dd80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } }
I20260501 13:52:57.985019 1470 leader_election.cc:290] T c6507fffd6a4466ab15b1eb89c7f32c7 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 election: Requested vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
I20260501 13:52:57.985852 1209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6507fffd6a4466ab15b1eb89c7f32c7" candidate_uuid: "e4a8a400d10343718a43d5fcc669dd80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1151db12ecae4dd7bb88b3d1415b7d92"
I20260501 13:52:57.986033 1209 raft_consensus.cc:3060] T c6507fffd6a4466ab15b1eb89c7f32c7 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Advancing to term 1
I20260501 13:52:57.987880 1465 tablet_bootstrap.cc:492] T 5c413244650e4784981922b6502f6b97 P e4a8a400d10343718a43d5fcc669dd80: Bootstrap starting.
I20260501 13:52:57.989059 1209 raft_consensus.cc:2468] T c6507fffd6a4466ab15b1eb89c7f32c7 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4a8a400d10343718a43d5fcc669dd80 in term 1.
I20260501 13:52:57.989513 1007 leader_election.cc:304] T c6507fffd6a4466ab15b1eb89c7f32c7 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1151db12ecae4dd7bb88b3d1415b7d92, e4a8a400d10343718a43d5fcc669dd80; no voters:
I20260501 13:52:57.989708 1465 tablet_bootstrap.cc:654] T 5c413244650e4784981922b6502f6b97 P e4a8a400d10343718a43d5fcc669dd80: Neither blocks nor log segments found. Creating new log.
I20260501 13:52:57.989863 1470 raft_consensus.cc:2804] T c6507fffd6a4466ab15b1eb89c7f32c7 P e4a8a400d10343718a43d5fcc669dd80 [term 1 FOLLOWER]: Leader election won for term 1
I20260501 13:52:57.993394 1464 tablet_bootstrap.cc:492] T 5c413244650e4784981922b6502f6b97 P b0226350eae044398f04ef4e5cbb550e: No bootstrap required, opened a new log
I20260501 13:52:57.993579 1464 ts_tablet_manager.cc:1403] T 5c413244650e4784981922b6502f6b97 P b0226350eae044398f04ef4e5cbb550e: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.007s
I20260501 13:52:57.994310 1464 raft_consensus.cc:359] T 5c413244650e4784981922b6502f6b97 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:57.994491 1464 raft_consensus.cc:385] T 5c413244650e4784981922b6502f6b97 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260501 13:52:57.994557 1464 raft_consensus.cc:740] T 5c413244650e4784981922b6502f6b97 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0226350eae044398f04ef4e5cbb550e, State: Initialized, Role: FOLLOWER
W20260501 13:52:57.995388 1119 tablet.cc:2404] T 546264ae15814277b0acfb77598562df P e4a8a400d10343718a43d5fcc669dd80: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260501 13:52:57.996873 1464 consensus_queue.cc:260] T 5c413244650e4784981922b6502f6b97 P b0226350eae044398f04ef4e5cbb550e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:57.997349 1343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6507fffd6a4466ab15b1eb89c7f32c7" candidate_uuid: "e4a8a400d10343718a43d5fcc669dd80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0226350eae044398f04ef4e5cbb550e" is_pre_election: true
I20260501 13:52:57.998164 1464 ts_tablet_manager.cc:1434] T 5c413244650e4784981922b6502f6b97 P b0226350eae044398f04ef4e5cbb550e: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260501 13:52:57.998311 1343 raft_consensus.cc:2468] T c6507fffd6a4466ab15b1eb89c7f32c7 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4a8a400d10343718a43d5fcc669dd80 in term 0.
I20260501 13:52:57.998682 1342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6507fffd6a4466ab15b1eb89c7f32c7" candidate_uuid: "e4a8a400d10343718a43d5fcc669dd80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0226350eae044398f04ef4e5cbb550e"
I20260501 13:52:57.998911 1342 raft_consensus.cc:3060] T c6507fffd6a4466ab15b1eb89c7f32c7 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Advancing to term 1
W20260501 13:52:58.002409 1257 tablet.cc:2404] T 546264ae15814277b0acfb77598562df P 1151db12ecae4dd7bb88b3d1415b7d92: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260501 13:52:58.003839 1470 raft_consensus.cc:697] T c6507fffd6a4466ab15b1eb89c7f32c7 P e4a8a400d10343718a43d5fcc669dd80 [term 1 LEADER]: Becoming Leader. State: Replica: e4a8a400d10343718a43d5fcc669dd80, State: Running, Role: LEADER
I20260501 13:52:58.004361 1470 consensus_queue.cc:237] T c6507fffd6a4466ab15b1eb89c7f32c7 P e4a8a400d10343718a43d5fcc669dd80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } }
I20260501 13:52:58.005613 727 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: Sending DeleteTablet for 3 replicas of tablet 546264ae15814277b0acfb77598562df
I20260501 13:52:58.006744 1051 tablet_service.cc:1558] Processing DeleteTablet for tablet 546264ae15814277b0acfb77598562df with delete_type TABLET_DATA_DELETED (replaced by tablet 3cb76051f80b49439f8b9191af3cef97) from {username='slave'} at 127.0.0.1:51140
I20260501 13:52:58.010658 1188 tablet_service.cc:1558] Processing DeleteTablet for tablet 546264ae15814277b0acfb77598562df with delete_type TABLET_DATA_DELETED (replaced by tablet 3cb76051f80b49439f8b9191af3cef97) from {username='slave'} at 127.0.0.1:33776
W20260501 13:52:58.014806 1391 tablet.cc:2404] T 546264ae15814277b0acfb77598562df P b0226350eae044398f04ef4e5cbb550e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260501 13:52:58.020146 1322 tablet_service.cc:1558] Processing DeleteTablet for tablet 546264ae15814277b0acfb77598562df with delete_type TABLET_DATA_DELETED (replaced by tablet 3cb76051f80b49439f8b9191af3cef97) from {username='slave'} at 127.0.0.1:37808
I20260501 13:52:58.020862 1465 tablet_bootstrap.cc:492] T 5c413244650e4784981922b6502f6b97 P e4a8a400d10343718a43d5fcc669dd80: No bootstrap required, opened a new log
I20260501 13:52:58.021049 1465 ts_tablet_manager.cc:1403] T 5c413244650e4784981922b6502f6b97 P e4a8a400d10343718a43d5fcc669dd80: Time spent bootstrapping tablet: real 0.033s user 0.008s sys 0.002s
I20260501 13:52:58.021876 1465 raft_consensus.cc:359] T 5c413244650e4784981922b6502f6b97 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:58.022078 1465 raft_consensus.cc:385] T 5c413244650e4784981922b6502f6b97 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260501 13:52:58.022174 1465 raft_consensus.cc:740] T 5c413244650e4784981922b6502f6b97 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4a8a400d10343718a43d5fcc669dd80, State: Initialized, Role: FOLLOWER
I20260501 13:52:58.022413 1465 consensus_queue.cc:260] T 5c413244650e4784981922b6502f6b97 P e4a8a400d10343718a43d5fcc669dd80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:58.023056 1465 ts_tablet_manager.cc:1434] T 5c413244650e4784981922b6502f6b97 P e4a8a400d10343718a43d5fcc669dd80: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260501 13:52:58.031744 1483 tablet_replica.cc:333] T 546264ae15814277b0acfb77598562df P 1151db12ecae4dd7bb88b3d1415b7d92: stopping tablet replica
I20260501 13:52:58.032187 1483 raft_consensus.cc:2243] T 546264ae15814277b0acfb77598562df P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260501 13:52:58.032581 1483 raft_consensus.cc:2272] T 546264ae15814277b0acfb77598562df P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260501 13:52:58.033987 1463 tablet_bootstrap.cc:492] T 5c413244650e4784981922b6502f6b97 P 1151db12ecae4dd7bb88b3d1415b7d92: No bootstrap required, opened a new log
I20260501 13:52:58.034178 1463 ts_tablet_manager.cc:1403] T 5c413244650e4784981922b6502f6b97 P 1151db12ecae4dd7bb88b3d1415b7d92: Time spent bootstrapping tablet: real 0.056s user 0.007s sys 0.004s
I20260501 13:52:58.034509 1189 tablet_service.cc:1511] Processing CreateTablet for tablet 3cb76051f80b49439f8b9191af3cef97 (DEFAULT_TABLE table=default.table_8beb55fb2bdd42309d0183432704b989 [id=f7226f45b9d94af2ad176c2d4a419e88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260501 13:52:58.034983 1189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cb76051f80b49439f8b9191af3cef97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260501 13:52:58.035053 1463 raft_consensus.cc:359] T 5c413244650e4784981922b6502f6b97 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:58.035326 1463 raft_consensus.cc:385] T 5c413244650e4784981922b6502f6b97 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260501 13:52:58.035449 1463 raft_consensus.cc:740] T 5c413244650e4784981922b6502f6b97 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1151db12ecae4dd7bb88b3d1415b7d92, State: Initialized, Role: FOLLOWER
I20260501 13:52:58.035778 1463 consensus_queue.cc:260] T 5c413244650e4784981922b6502f6b97 P 1151db12ecae4dd7bb88b3d1415b7d92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:58.038043 1463 ts_tablet_manager.cc:1434] T 5c413244650e4784981922b6502f6b97 P 1151db12ecae4dd7bb88b3d1415b7d92: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260501 13:52:58.039698 1483 ts_tablet_manager.cc:1916] T 546264ae15814277b0acfb77598562df P 1151db12ecae4dd7bb88b3d1415b7d92: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260501 13:52:58.043406 727 catalog_manager.cc:6741] ReplaceTablet: tablet 546264ae15814277b0acfb77598562df deleted and replaced by tablet 3cb76051f80b49439f8b9191af3cef97
I20260501 13:52:58.044622 1050 tablet_service.cc:1511] Processing CreateTablet for tablet 3cb76051f80b49439f8b9191af3cef97 (DEFAULT_TABLE table=default.table_8beb55fb2bdd42309d0183432704b989 [id=f7226f45b9d94af2ad176c2d4a419e88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260501 13:52:58.045346 1050 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cb76051f80b49439f8b9191af3cef97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260501 13:52:58.048390 1463 tablet_bootstrap.cc:492] T 3cb76051f80b49439f8b9191af3cef97 P 1151db12ecae4dd7bb88b3d1415b7d92: Bootstrap starting.
I20260501 13:52:58.048383 1323 tablet_service.cc:1511] Processing CreateTablet for tablet 3cb76051f80b49439f8b9191af3cef97 (DEFAULT_TABLE table=default.table_8beb55fb2bdd42309d0183432704b989 [id=f7226f45b9d94af2ad176c2d4a419e88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260501 13:52:58.048837 1323 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cb76051f80b49439f8b9191af3cef97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260501 13:52:58.052695 1484 tablet_replica.cc:333] T 546264ae15814277b0acfb77598562df P b0226350eae044398f04ef4e5cbb550e: stopping tablet replica
I20260501 13:52:58.053292 1484 raft_consensus.cc:2243] T 546264ae15814277b0acfb77598562df P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Raft consensus shutting down.
I20260501 13:52:58.053758 1484 raft_consensus.cc:2272] T 546264ae15814277b0acfb77598562df P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Raft consensus is shut down!
I20260501 13:52:58.055418 1484 ts_tablet_manager.cc:1916] T 546264ae15814277b0acfb77598562df P b0226350eae044398f04ef4e5cbb550e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260501 13:52:58.058876 1481 tablet_replica.cc:333] T 546264ae15814277b0acfb77598562df P e4a8a400d10343718a43d5fcc669dd80: stopping tablet replica
I20260501 13:52:58.059514 1465 tablet_bootstrap.cc:492] T 3cb76051f80b49439f8b9191af3cef97 P e4a8a400d10343718a43d5fcc669dd80: Bootstrap starting.
I20260501 13:52:58.060374 1463 tablet_bootstrap.cc:654] T 3cb76051f80b49439f8b9191af3cef97 P 1151db12ecae4dd7bb88b3d1415b7d92: Neither blocks nor log segments found. Creating new log.
I20260501 13:52:58.061165 1465 tablet_bootstrap.cc:654] T 3cb76051f80b49439f8b9191af3cef97 P e4a8a400d10343718a43d5fcc669dd80: Neither blocks nor log segments found. Creating new log.
I20260501 13:52:58.064760 1481 raft_consensus.cc:2243] T 546264ae15814277b0acfb77598562df P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260501 13:52:58.065203 1481 raft_consensus.cc:2272] T 546264ae15814277b0acfb77598562df P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260501 13:52:58.065609 727 catalog_manager.cc:5671] T c6507fffd6a4466ab15b1eb89c7f32c7 P e4a8a400d10343718a43d5fcc669dd80 reported cstate change: term changed from 0 to 1, leader changed from <none> to e4a8a400d10343718a43d5fcc669dd80 (127.0.105.1). New cstate: current_term: 1 leader_uuid: "e4a8a400d10343718a43d5fcc669dd80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } health_report { overall_health: UNKNOWN } } }
I20260501 13:52:58.068353 1481 ts_tablet_manager.cc:1916] T 546264ae15814277b0acfb77598562df P e4a8a400d10343718a43d5fcc669dd80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260501 13:52:58.068868 1463 tablet_bootstrap.cc:492] T 3cb76051f80b49439f8b9191af3cef97 P 1151db12ecae4dd7bb88b3d1415b7d92: No bootstrap required, opened a new log
I20260501 13:52:58.069073 1463 ts_tablet_manager.cc:1403] T 3cb76051f80b49439f8b9191af3cef97 P 1151db12ecae4dd7bb88b3d1415b7d92: Time spent bootstrapping tablet: real 0.021s user 0.006s sys 0.004s
I20260501 13:52:58.069941 1463 raft_consensus.cc:359] T 3cb76051f80b49439f8b9191af3cef97 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:58.070199 1463 raft_consensus.cc:385] T 3cb76051f80b49439f8b9191af3cef97 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260501 13:52:58.070326 1463 raft_consensus.cc:740] T 3cb76051f80b49439f8b9191af3cef97 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1151db12ecae4dd7bb88b3d1415b7d92, State: Initialized, Role: FOLLOWER
I20260501 13:52:58.070657 1463 consensus_queue.cc:260] T 3cb76051f80b49439f8b9191af3cef97 P 1151db12ecae4dd7bb88b3d1415b7d92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:58.072952 1463 ts_tablet_manager.cc:1434] T 3cb76051f80b49439f8b9191af3cef97 P 1151db12ecae4dd7bb88b3d1415b7d92: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260501 13:52:58.078104 1473 raft_consensus.cc:493] T 3cb76051f80b49439f8b9191af3cef97 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:52:58.078269 1473 raft_consensus.cc:515] T 3cb76051f80b49439f8b9191af3cef97 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:58.080121 1473 leader_election.cc:290] T 3cb76051f80b49439f8b9191af3cef97 P 1151db12ecae4dd7bb88b3d1415b7d92 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e4a8a400d10343718a43d5fcc669dd80 (127.0.105.1:44349), b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841)
I20260501 13:52:58.083648 1342 raft_consensus.cc:2468] T c6507fffd6a4466ab15b1eb89c7f32c7 P b0226350eae044398f04ef4e5cbb550e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4a8a400d10343718a43d5fcc669dd80 in term 1.
I20260501 13:52:58.086880 1464 tablet_bootstrap.cc:492] T 3cb76051f80b49439f8b9191af3cef97 P b0226350eae044398f04ef4e5cbb550e: Bootstrap starting.
I20260501 13:52:58.088696 1464 tablet_bootstrap.cc:654] T 3cb76051f80b49439f8b9191af3cef97 P b0226350eae044398f04ef4e5cbb550e: Neither blocks nor log segments found. Creating new log.
I20260501 13:52:58.090342 727 master_service.cc:954] ReplaceTablet: received request to replace tablet 546264ae15814277b0acfb77598562df from {username='slave'} at 127.0.0.1:35974
I20260501 13:52:58.091488 727 master_service.cc:954] ReplaceTablet: received request to replace tablet 546264ae15814277b0acfb77598562df from {username='slave'} at 127.0.0.1:35974
I20260501 13:52:58.091704 1483 ts_tablet_manager.cc:1929] T 546264ae15814277b0acfb77598562df P 1151db12ecae4dd7bb88b3d1415b7d92: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260501 13:52:58.091903 1483 log.cc:1199] T 546264ae15814277b0acfb77598562df P 1151db12ecae4dd7bb88b3d1415b7d92: Deleting WAL directory at /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-1/wal/wals/546264ae15814277b0acfb77598562df
I20260501 13:52:58.091895 730 catalog_manager.cc:2174] created HMS catalog entry for table [id=ded9bef73cb942de81c7b635d42480f8]
I20260501 13:52:58.092367 727 master_service.cc:954] ReplaceTablet: received request to replace tablet 546264ae15814277b0acfb77598562df from {username='slave'} at 127.0.0.1:35974
I20260501 13:52:58.092423 1483 ts_tablet_manager.cc:1950] T 546264ae15814277b0acfb77598562df P 1151db12ecae4dd7bb88b3d1415b7d92: Deleting consensus metadata
I20260501 13:52:58.093168 727 master_service.cc:954] ReplaceTablet: received request to replace tablet c6507fffd6a4466ab15b1eb89c7f32c7 from {username='slave'} at 127.0.0.1:35974
I20260501 13:52:58.097489 1465 tablet_bootstrap.cc:492] T 3cb76051f80b49439f8b9191af3cef97 P e4a8a400d10343718a43d5fcc669dd80: No bootstrap required, opened a new log
I20260501 13:52:58.097693 1465 ts_tablet_manager.cc:1403] T 3cb76051f80b49439f8b9191af3cef97 P e4a8a400d10343718a43d5fcc669dd80: Time spent bootstrapping tablet: real 0.038s user 0.005s sys 0.005s
I20260501 13:52:58.098621 1465 raft_consensus.cc:359] T 3cb76051f80b49439f8b9191af3cef97 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:58.098825 1465 raft_consensus.cc:385] T 3cb76051f80b49439f8b9191af3cef97 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260501 13:52:58.098913 1465 raft_consensus.cc:740] T 3cb76051f80b49439f8b9191af3cef97 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4a8a400d10343718a43d5fcc669dd80, State: Initialized, Role: FOLLOWER
I20260501 13:52:58.099159 1465 consensus_queue.cc:260] T 3cb76051f80b49439f8b9191af3cef97 P e4a8a400d10343718a43d5fcc669dd80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:58.099759 1465 ts_tablet_manager.cc:1434] T 3cb76051f80b49439f8b9191af3cef97 P e4a8a400d10343718a43d5fcc669dd80: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260501 13:52:58.105413 1464 tablet_bootstrap.cc:492] T 3cb76051f80b49439f8b9191af3cef97 P b0226350eae044398f04ef4e5cbb550e: No bootstrap required, opened a new log
I20260501 13:52:58.105588 1464 ts_tablet_manager.cc:1403] T 3cb76051f80b49439f8b9191af3cef97 P b0226350eae044398f04ef4e5cbb550e: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.006s
I20260501 13:52:58.106537 1464 raft_consensus.cc:359] T 3cb76051f80b49439f8b9191af3cef97 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:58.106737 1464 raft_consensus.cc:385] T 3cb76051f80b49439f8b9191af3cef97 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260501 13:52:58.106832 1464 raft_consensus.cc:740] T 3cb76051f80b49439f8b9191af3cef97 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0226350eae044398f04ef4e5cbb550e, State: Initialized, Role: FOLLOWER
I20260501 13:52:58.107084 1464 consensus_queue.cc:260] T 3cb76051f80b49439f8b9191af3cef97 P b0226350eae044398f04ef4e5cbb550e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:58.111799 1464 ts_tablet_manager.cc:1434] T 3cb76051f80b49439f8b9191af3cef97 P b0226350eae044398f04ef4e5cbb550e: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260501 13:52:58.120294 723 catalog_manager.cc:5002] TS 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): tablet 546264ae15814277b0acfb77598562df (table default.table_8beb55fb2bdd42309d0183432704b989 [id=f7226f45b9d94af2ad176c2d4a419e88]) successfully deleted
I20260501 13:52:58.122335 1071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb76051f80b49439f8b9191af3cef97" candidate_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4a8a400d10343718a43d5fcc669dd80" is_pre_election: true
I20260501 13:52:58.122763 1071 raft_consensus.cc:2468] T 3cb76051f80b49439f8b9191af3cef97 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1151db12ecae4dd7bb88b3d1415b7d92 in term 0.
I20260501 13:52:58.123446 1144 leader_election.cc:304] T 3cb76051f80b49439f8b9191af3cef97 P 1151db12ecae4dd7bb88b3d1415b7d92 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1151db12ecae4dd7bb88b3d1415b7d92, e4a8a400d10343718a43d5fcc669dd80; no voters:
I20260501 13:52:58.123880 1473 raft_consensus.cc:2804] T 3cb76051f80b49439f8b9191af3cef97 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260501 13:52:58.123998 1473 raft_consensus.cc:493] T 3cb76051f80b49439f8b9191af3cef97 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260501 13:52:58.124084 1473 raft_consensus.cc:3060] T 3cb76051f80b49439f8b9191af3cef97 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Advancing to term 1
I20260501 13:52:58.124923 1342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb76051f80b49439f8b9191af3cef97" candidate_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0226350eae044398f04ef4e5cbb550e" is_pre_election: true
I20260501 13:52:58.125152 1342 raft_consensus.cc:2468] T 3cb76051f80b49439f8b9191af3cef97 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1151db12ecae4dd7bb88b3d1415b7d92 in term 0.
I20260501 13:52:58.127619 727 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: Sending DeleteTablet for 3 replicas of tablet c6507fffd6a4466ab15b1eb89c7f32c7
I20260501 13:52:58.130709 1050 tablet_service.cc:1558] Processing DeleteTablet for tablet c6507fffd6a4466ab15b1eb89c7f32c7 with delete_type TABLET_DATA_DELETED (replaced by tablet daa28e4cd0d042b5bb7f7ea04ddb19b7) from {username='slave'} at 127.0.0.1:51140
I20260501 13:52:58.132010 727 catalog_manager.cc:6741] ReplaceTablet: tablet c6507fffd6a4466ab15b1eb89c7f32c7 deleted and replaced by tablet daa28e4cd0d042b5bb7f7ea04ddb19b7
I20260501 13:52:58.132380 1323 tablet_service.cc:1558] Processing DeleteTablet for tablet c6507fffd6a4466ab15b1eb89c7f32c7 with delete_type TABLET_DATA_DELETED (replaced by tablet daa28e4cd0d042b5bb7f7ea04ddb19b7) from {username='slave'} at 127.0.0.1:37808
I20260501 13:52:58.133356 1188 tablet_service.cc:1558] Processing DeleteTablet for tablet c6507fffd6a4466ab15b1eb89c7f32c7 with delete_type TABLET_DATA_DELETED (replaced by tablet daa28e4cd0d042b5bb7f7ea04ddb19b7) from {username='slave'} at 127.0.0.1:33776
I20260501 13:52:58.133661 1483 tablet_replica.cc:333] T c6507fffd6a4466ab15b1eb89c7f32c7 P 1151db12ecae4dd7bb88b3d1415b7d92: stopping tablet replica
I20260501 13:52:58.133826 1483 raft_consensus.cc:2243] T c6507fffd6a4466ab15b1eb89c7f32c7 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260501 13:52:58.133950 1483 raft_consensus.cc:2272] T c6507fffd6a4466ab15b1eb89c7f32c7 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260501 13:52:58.134944 1483 ts_tablet_manager.cc:1916] T c6507fffd6a4466ab15b1eb89c7f32c7 P 1151db12ecae4dd7bb88b3d1415b7d92: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260501 13:52:58.143164 1323 tablet_service.cc:1511] Processing CreateTablet for tablet 9fc54df2e5e14019af12a8c0977f967f (DEFAULT_TABLE table=default.table_06eba6fa1b4949ffa139c461e70349f9 [id=ded9bef73cb942de81c7b635d42480f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260501 13:52:58.143150 730 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35974:
table { table_name: "default.Table_293c891afe6c4d609f8069858909d21b" } new_table_name: "default.Table_06273705ee5b4dff82f6d1ad5c14fac4" modify_external_catalogs: true
I20260501 13:52:58.145028 727 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35974:
name: "default.Table_058c3f74f9b04b9d92dcd4667e109a5d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260501 13:52:58.146559 1050 tablet_service.cc:1511] Processing CreateTablet for tablet 9fc54df2e5e14019af12a8c0977f967f (DEFAULT_TABLE table=default.table_06eba6fa1b4949ffa139c461e70349f9 [id=ded9bef73cb942de81c7b635d42480f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260501 13:52:58.147300 1188 tablet_service.cc:1511] Processing CreateTablet for tablet 9fc54df2e5e14019af12a8c0977f967f (DEFAULT_TABLE table=default.table_06eba6fa1b4949ffa139c461e70349f9 [id=ded9bef73cb942de81c7b635d42480f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260501 13:52:58.148507 1189 tablet_service.cc:1511] Processing CreateTablet for tablet daa28e4cd0d042b5bb7f7ea04ddb19b7 (DEFAULT_TABLE table=default.table_5a0a455cca3c4ab6a059aac962b7cea0 [id=8818494b487941a4877edf5b079d9756]), partition=RANGE (key) PARTITION UNBOUNDED
I20260501 13:52:58.148756 1322 tablet_service.cc:1511] Processing CreateTablet for tablet daa28e4cd0d042b5bb7f7ea04ddb19b7 (DEFAULT_TABLE table=default.table_5a0a455cca3c4ab6a059aac962b7cea0 [id=8818494b487941a4877edf5b079d9756]), partition=RANGE (key) PARTITION UNBOUNDED
I20260501 13:52:58.149593 1051 tablet_service.cc:1511] Processing CreateTablet for tablet daa28e4cd0d042b5bb7f7ea04ddb19b7 (DEFAULT_TABLE table=default.table_5a0a455cca3c4ab6a059aac962b7cea0 [id=8818494b487941a4877edf5b079d9756]), partition=RANGE (key) PARTITION UNBOUNDED
I20260501 13:52:58.173636 1484 ts_tablet_manager.cc:1929] T 546264ae15814277b0acfb77598562df P b0226350eae044398f04ef4e5cbb550e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260501 13:52:58.173833 1484 log.cc:1199] T 546264ae15814277b0acfb77598562df P b0226350eae044398f04ef4e5cbb550e: Deleting WAL directory at /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-2/wal/wals/546264ae15814277b0acfb77598562df
I20260501 13:52:58.174268 1484 ts_tablet_manager.cc:1950] T 546264ae15814277b0acfb77598562df P b0226350eae044398f04ef4e5cbb550e: Deleting consensus metadata
I20260501 13:52:58.174741 1473 raft_consensus.cc:515] T 3cb76051f80b49439f8b9191af3cef97 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:58.175508 1473 leader_election.cc:290] T 3cb76051f80b49439f8b9191af3cef97 P 1151db12ecae4dd7bb88b3d1415b7d92 [CANDIDATE]: Term 1 election: Requested vote from peers e4a8a400d10343718a43d5fcc669dd80 (127.0.105.1:44349), b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841)
I20260501 13:52:58.175879 1323 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fc54df2e5e14019af12a8c0977f967f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260501 13:52:58.176461 1322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet daa28e4cd0d042b5bb7f7ea04ddb19b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260501 13:52:58.179891 1050 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fc54df2e5e14019af12a8c0977f967f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260501 13:52:58.180462 1051 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet daa28e4cd0d042b5bb7f7ea04ddb19b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260501 13:52:58.187009 1189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet daa28e4cd0d042b5bb7f7ea04ddb19b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260501 13:52:58.187250 1071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb76051f80b49439f8b9191af3cef97" candidate_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4a8a400d10343718a43d5fcc669dd80"
I20260501 13:52:58.187458 1071 raft_consensus.cc:3060] T 3cb76051f80b49439f8b9191af3cef97 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Advancing to term 1
I20260501 13:52:58.187808 1188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fc54df2e5e14019af12a8c0977f967f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260501 13:52:58.195462 1342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb76051f80b49439f8b9191af3cef97" candidate_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0226350eae044398f04ef4e5cbb550e"
I20260501 13:52:58.195724 1342 raft_consensus.cc:3060] T 3cb76051f80b49439f8b9191af3cef97 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Advancing to term 1
I20260501 13:52:58.198385 1464 tablet_bootstrap.cc:492] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P b0226350eae044398f04ef4e5cbb550e: Bootstrap starting.
I20260501 13:52:58.200237 1464 tablet_bootstrap.cc:654] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P b0226350eae044398f04ef4e5cbb550e: Neither blocks nor log segments found. Creating new log.
I20260501 13:52:58.201560 1463 tablet_bootstrap.cc:492] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92: Bootstrap starting.
I20260501 13:52:58.203487 1463 tablet_bootstrap.cc:654] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92: Neither blocks nor log segments found. Creating new log.
I20260501 13:52:58.213447 1464 tablet_bootstrap.cc:492] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P b0226350eae044398f04ef4e5cbb550e: No bootstrap required, opened a new log
I20260501 13:52:58.214277 1464 ts_tablet_manager.cc:1403] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P b0226350eae044398f04ef4e5cbb550e: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.006s
I20260501 13:52:58.215060 1463 tablet_bootstrap.cc:492] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92: No bootstrap required, opened a new log
I20260501 13:52:58.215281 1463 ts_tablet_manager.cc:1403] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.011s
I20260501 13:52:58.215318 1464 raft_consensus.cc:359] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:58.215675 1464 raft_consensus.cc:385] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260501 13:52:58.215797 1464 raft_consensus.cc:740] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0226350eae044398f04ef4e5cbb550e, State: Initialized, Role: FOLLOWER
I20260501 13:52:58.216145 1463 raft_consensus.cc:359] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:58.216141 1464 consensus_queue.cc:260] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P b0226350eae044398f04ef4e5cbb550e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:58.216336 1463 raft_consensus.cc:385] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260501 13:52:58.216403 1463 raft_consensus.cc:740] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1151db12ecae4dd7bb88b3d1415b7d92, State: Initialized, Role: FOLLOWER
I20260501 13:52:58.216688 1463 consensus_queue.cc:260] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:58.218263 1464 ts_tablet_manager.cc:1434] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P b0226350eae044398f04ef4e5cbb550e: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260501 13:52:58.220028 1463 ts_tablet_manager.cc:1434] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260501 13:52:58.231469 1469 raft_consensus.cc:493] T 5c413244650e4784981922b6502f6b97 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:52:58.231700 1469 raft_consensus.cc:515] T 5c413244650e4784981922b6502f6b97 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:58.233783 1469 leader_election.cc:290] T 5c413244650e4784981922b6502f6b97 P b0226350eae044398f04ef4e5cbb550e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e4a8a400d10343718a43d5fcc669dd80 (127.0.105.1:44349), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
I20260501 13:52:58.243451 1070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c413244650e4784981922b6502f6b97" candidate_uuid: "b0226350eae044398f04ef4e5cbb550e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4a8a400d10343718a43d5fcc669dd80" is_pre_election: true
I20260501 13:52:58.243865 1070 raft_consensus.cc:2468] T 5c413244650e4784981922b6502f6b97 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0226350eae044398f04ef4e5cbb550e in term 0.
I20260501 13:52:58.244632 1278 leader_election.cc:304] T 5c413244650e4784981922b6502f6b97 P b0226350eae044398f04ef4e5cbb550e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0226350eae044398f04ef4e5cbb550e, e4a8a400d10343718a43d5fcc669dd80; no voters:
I20260501 13:52:58.247881 1469 raft_consensus.cc:2804] T 5c413244650e4784981922b6502f6b97 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260501 13:52:58.248014 1469 raft_consensus.cc:493] T 5c413244650e4784981922b6502f6b97 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260501 13:52:58.248106 1469 raft_consensus.cc:3060] T 5c413244650e4784981922b6502f6b97 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Advancing to term 1
I20260501 13:52:58.251319 1209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c413244650e4784981922b6502f6b97" candidate_uuid: "b0226350eae044398f04ef4e5cbb550e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" is_pre_election: true
I20260501 13:52:58.251576 1209 raft_consensus.cc:2468] T 5c413244650e4784981922b6502f6b97 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0226350eae044398f04ef4e5cbb550e in term 0.
I20260501 13:52:58.267097 1342 raft_consensus.cc:2468] T 3cb76051f80b49439f8b9191af3cef97 P b0226350eae044398f04ef4e5cbb550e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1151db12ecae4dd7bb88b3d1415b7d92 in term 1.
I20260501 13:52:58.267733 1481 ts_tablet_manager.cc:1929] T 546264ae15814277b0acfb77598562df P e4a8a400d10343718a43d5fcc669dd80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260501 13:52:58.267920 1481 log.cc:1199] T 546264ae15814277b0acfb77598562df P e4a8a400d10343718a43d5fcc669dd80: Deleting WAL directory at /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-0/wal/wals/546264ae15814277b0acfb77598562df
I20260501 13:52:58.268316 1484 tablet_replica.cc:333] T c6507fffd6a4466ab15b1eb89c7f32c7 P b0226350eae044398f04ef4e5cbb550e: stopping tablet replica
I20260501 13:52:58.268359 1481 ts_tablet_manager.cc:1950] T 546264ae15814277b0acfb77598562df P e4a8a400d10343718a43d5fcc669dd80: Deleting consensus metadata
I20260501 13:52:58.268527 1484 raft_consensus.cc:2243] T c6507fffd6a4466ab15b1eb89c7f32c7 P b0226350eae044398f04ef4e5cbb550e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260501 13:52:58.268707 1484 raft_consensus.cc:2272] T c6507fffd6a4466ab15b1eb89c7f32c7 P b0226350eae044398f04ef4e5cbb550e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260501 13:52:58.269052 1071 raft_consensus.cc:2468] T 3cb76051f80b49439f8b9191af3cef97 P e4a8a400d10343718a43d5fcc669dd80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1151db12ecae4dd7bb88b3d1415b7d92 in term 1.
I20260501 13:52:58.269538 1483 ts_tablet_manager.cc:1929] T c6507fffd6a4466ab15b1eb89c7f32c7 P 1151db12ecae4dd7bb88b3d1415b7d92: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260501 13:52:58.269690 1483 log.cc:1199] T c6507fffd6a4466ab15b1eb89c7f32c7 P 1151db12ecae4dd7bb88b3d1415b7d92: Deleting WAL directory at /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-1/wal/wals/c6507fffd6a4466ab15b1eb89c7f32c7
I20260501 13:52:58.270028 1483 ts_tablet_manager.cc:1950] T c6507fffd6a4466ab15b1eb89c7f32c7 P 1151db12ecae4dd7bb88b3d1415b7d92: Deleting consensus metadata
I20260501 13:52:58.273100 1144 leader_election.cc:304] T 3cb76051f80b49439f8b9191af3cef97 P 1151db12ecae4dd7bb88b3d1415b7d92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1151db12ecae4dd7bb88b3d1415b7d92, e4a8a400d10343718a43d5fcc669dd80; no voters:
I20260501 13:52:58.273782 1473 raft_consensus.cc:2804] T 3cb76051f80b49439f8b9191af3cef97 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 1 FOLLOWER]: Leader election won for term 1
I20260501 13:52:58.276942 1484 ts_tablet_manager.cc:1916] T c6507fffd6a4466ab15b1eb89c7f32c7 P b0226350eae044398f04ef4e5cbb550e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260501 13:52:58.277477 1465 tablet_bootstrap.cc:492] T 9fc54df2e5e14019af12a8c0977f967f P e4a8a400d10343718a43d5fcc669dd80: Bootstrap starting.
I20260501 13:52:58.279577 1465 tablet_bootstrap.cc:654] T 9fc54df2e5e14019af12a8c0977f967f P e4a8a400d10343718a43d5fcc669dd80: Neither blocks nor log segments found. Creating new log.
I20260501 13:52:58.281122 721 catalog_manager.cc:5002] TS b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): tablet 546264ae15814277b0acfb77598562df (table default.table_8beb55fb2bdd42309d0183432704b989 [id=f7226f45b9d94af2ad176c2d4a419e88]) successfully deleted
I20260501 13:52:58.282938 1463 tablet_bootstrap.cc:492] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P 1151db12ecae4dd7bb88b3d1415b7d92: Bootstrap starting.
I20260501 13:52:58.285149 1463 tablet_bootstrap.cc:654] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P 1151db12ecae4dd7bb88b3d1415b7d92: Neither blocks nor log segments found. Creating new log.
I20260501 13:52:58.286775 1481 tablet_replica.cc:333] T c6507fffd6a4466ab15b1eb89c7f32c7 P e4a8a400d10343718a43d5fcc669dd80: stopping tablet replica
I20260501 13:52:58.287039 1481 raft_consensus.cc:2243] T c6507fffd6a4466ab15b1eb89c7f32c7 P e4a8a400d10343718a43d5fcc669dd80 [term 1 LEADER]: Raft consensus shutting down.
I20260501 13:52:58.287588 1481 pending_rounds.cc:70] T c6507fffd6a4466ab15b1eb89c7f32c7 P e4a8a400d10343718a43d5fcc669dd80: Trying to abort 1 pending ops.
I20260501 13:52:58.287566 722 catalog_manager.cc:5002] TS e4a8a400d10343718a43d5fcc669dd80 (127.0.105.1:44349): tablet 546264ae15814277b0acfb77598562df (table default.table_8beb55fb2bdd42309d0183432704b989 [id=f7226f45b9d94af2ad176c2d4a419e88]) successfully deleted
I20260501 13:52:58.287820 1481 pending_rounds.cc:77] T c6507fffd6a4466ab15b1eb89c7f32c7 P e4a8a400d10343718a43d5fcc669dd80: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7281228095511052288 op_type: NO_OP noop_request { }
I20260501 13:52:58.288031 1481 raft_consensus.cc:2889] T c6507fffd6a4466ab15b1eb89c7f32c7 P e4a8a400d10343718a43d5fcc669dd80 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260501 13:52:58.288189 1481 raft_consensus.cc:2272] T c6507fffd6a4466ab15b1eb89c7f32c7 P e4a8a400d10343718a43d5fcc669dd80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260501 13:52:58.289494 1481 ts_tablet_manager.cc:1916] T c6507fffd6a4466ab15b1eb89c7f32c7 P e4a8a400d10343718a43d5fcc669dd80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260501 13:52:58.292073 1465 tablet_bootstrap.cc:492] T 9fc54df2e5e14019af12a8c0977f967f P e4a8a400d10343718a43d5fcc669dd80: No bootstrap required, opened a new log
I20260501 13:52:58.292270 1465 ts_tablet_manager.cc:1403] T 9fc54df2e5e14019af12a8c0977f967f P e4a8a400d10343718a43d5fcc669dd80: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.000s
I20260501 13:52:58.293602 1465 raft_consensus.cc:359] T 9fc54df2e5e14019af12a8c0977f967f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:58.294121 1465 raft_consensus.cc:385] T 9fc54df2e5e14019af12a8c0977f967f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260501 13:52:58.294214 1465 raft_consensus.cc:740] T 9fc54df2e5e14019af12a8c0977f967f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4a8a400d10343718a43d5fcc669dd80, State: Initialized, Role: FOLLOWER
I20260501 13:52:58.294176 1493 raft_consensus.cc:493] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:52:58.294503 1465 consensus_queue.cc:260] T 9fc54df2e5e14019af12a8c0977f967f P e4a8a400d10343718a43d5fcc669dd80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:58.294459 1493 raft_consensus.cc:515] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:58.295642 1493 leader_election.cc:290] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e4a8a400d10343718a43d5fcc669dd80 (127.0.105.1:44349), b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841)
I20260501 13:52:58.296619 1473 raft_consensus.cc:697] T 3cb76051f80b49439f8b9191af3cef97 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 1 LEADER]: Becoming Leader. State: Replica: 1151db12ecae4dd7bb88b3d1415b7d92, State: Running, Role: LEADER
I20260501 13:52:58.296587 1071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc54df2e5e14019af12a8c0977f967f" candidate_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4a8a400d10343718a43d5fcc669dd80" is_pre_election: true
I20260501 13:52:58.296818 1071 raft_consensus.cc:2468] T 9fc54df2e5e14019af12a8c0977f967f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1151db12ecae4dd7bb88b3d1415b7d92 in term 0.
I20260501 13:52:58.297049 1473 consensus_queue.cc:237] T 3cb76051f80b49439f8b9191af3cef97 P 1151db12ecae4dd7bb88b3d1415b7d92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:58.297369 1144 leader_election.cc:304] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1151db12ecae4dd7bb88b3d1415b7d92, e4a8a400d10343718a43d5fcc669dd80; no voters:
I20260501 13:52:58.297771 1493 raft_consensus.cc:2804] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260501 13:52:58.297924 1493 raft_consensus.cc:493] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260501 13:52:58.298027 1493 raft_consensus.cc:3060] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Advancing to term 1
I20260501 13:52:58.298561 1465 ts_tablet_manager.cc:1434] T 9fc54df2e5e14019af12a8c0977f967f P e4a8a400d10343718a43d5fcc669dd80: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260501 13:52:58.300750 1465 tablet_bootstrap.cc:492] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P e4a8a400d10343718a43d5fcc669dd80: Bootstrap starting.
W20260501 13:52:58.300858 1143 leader_election.cc:343] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: 9fc54df2e5e14019af12a8c0977f967f
I20260501 13:52:58.302544 1465 tablet_bootstrap.cc:654] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P e4a8a400d10343718a43d5fcc669dd80: Neither blocks nor log segments found. Creating new log.
I20260501 13:52:58.306797 1463 tablet_bootstrap.cc:492] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P 1151db12ecae4dd7bb88b3d1415b7d92: No bootstrap required, opened a new log
I20260501 13:52:58.307161 1463 ts_tablet_manager.cc:1403] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P 1151db12ecae4dd7bb88b3d1415b7d92: Time spent bootstrapping tablet: real 0.024s user 0.010s sys 0.001s
I20260501 13:52:58.308348 1463 raft_consensus.cc:359] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:58.308743 1463 raft_consensus.cc:385] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260501 13:52:58.308916 1463 raft_consensus.cc:740] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1151db12ecae4dd7bb88b3d1415b7d92, State: Initialized, Role: FOLLOWER
I20260501 13:52:58.309300 1463 consensus_queue.cc:260] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P 1151db12ecae4dd7bb88b3d1415b7d92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:58.320058 1463 ts_tablet_manager.cc:1434] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P 1151db12ecae4dd7bb88b3d1415b7d92: Time spent starting tablet: real 0.013s user 0.005s sys 0.006s
I20260501 13:52:58.320951 1465 tablet_bootstrap.cc:492] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P e4a8a400d10343718a43d5fcc669dd80: No bootstrap required, opened a new log
I20260501 13:52:58.321058 723 catalog_manager.cc:5002] TS 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): tablet c6507fffd6a4466ab15b1eb89c7f32c7 (table default.table_5a0a455cca3c4ab6a059aac962b7cea0 [id=8818494b487941a4877edf5b079d9756]) successfully deleted
I20260501 13:52:58.321151 1465 ts_tablet_manager.cc:1403] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P e4a8a400d10343718a43d5fcc669dd80: Time spent bootstrapping tablet: real 0.021s user 0.005s sys 0.012s
I20260501 13:52:58.322731 1465 raft_consensus.cc:359] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:58.322932 1465 raft_consensus.cc:385] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260501 13:52:58.323010 1465 raft_consensus.cc:740] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4a8a400d10343718a43d5fcc669dd80, State: Initialized, Role: FOLLOWER
I20260501 13:52:58.323263 1465 consensus_queue.cc:260] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P e4a8a400d10343718a43d5fcc669dd80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:58.323750 1465 ts_tablet_manager.cc:1434] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P e4a8a400d10343718a43d5fcc669dd80: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260501 13:52:58.354135 1469 raft_consensus.cc:515] T 5c413244650e4784981922b6502f6b97 P b0226350eae044398f04ef4e5cbb550e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:58.355163 1469 leader_election.cc:290] T 5c413244650e4784981922b6502f6b97 P b0226350eae044398f04ef4e5cbb550e [CANDIDATE]: Term 1 election: Requested vote from peers e4a8a400d10343718a43d5fcc669dd80 (127.0.105.1:44349), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
I20260501 13:52:58.355706 1493 raft_consensus.cc:515] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:58.356559 1493 leader_election.cc:290] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92 [CANDIDATE]: Term 1 election: Requested vote from peers e4a8a400d10343718a43d5fcc669dd80 (127.0.105.1:44349), b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841)
I20260501 13:52:58.357767 1484 ts_tablet_manager.cc:1929] T c6507fffd6a4466ab15b1eb89c7f32c7 P b0226350eae044398f04ef4e5cbb550e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260501 13:52:58.357919 1484 log.cc:1199] T c6507fffd6a4466ab15b1eb89c7f32c7 P b0226350eae044398f04ef4e5cbb550e: Deleting WAL directory at /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-2/wal/wals/c6507fffd6a4466ab15b1eb89c7f32c7
I20260501 13:52:58.358297 1484 ts_tablet_manager.cc:1950] T c6507fffd6a4466ab15b1eb89c7f32c7 P b0226350eae044398f04ef4e5cbb550e: Deleting consensus metadata
I20260501 13:52:58.358260 1071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c413244650e4784981922b6502f6b97" candidate_uuid: "b0226350eae044398f04ef4e5cbb550e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4a8a400d10343718a43d5fcc669dd80"
I20260501 13:52:58.358525 1071 raft_consensus.cc:3060] T 5c413244650e4784981922b6502f6b97 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Advancing to term 1
I20260501 13:52:58.360997 1342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc54df2e5e14019af12a8c0977f967f" candidate_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0226350eae044398f04ef4e5cbb550e"
I20260501 13:52:58.361893 1464 tablet_bootstrap.cc:492] T 9fc54df2e5e14019af12a8c0977f967f P b0226350eae044398f04ef4e5cbb550e: Bootstrap starting.
I20260501 13:52:58.362027 1209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c413244650e4784981922b6502f6b97" candidate_uuid: "b0226350eae044398f04ef4e5cbb550e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1151db12ecae4dd7bb88b3d1415b7d92"
I20260501 13:52:58.362277 1209 raft_consensus.cc:3060] T 5c413244650e4784981922b6502f6b97 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Advancing to term 1
I20260501 13:52:58.362560 1070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc54df2e5e14019af12a8c0977f967f" candidate_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4a8a400d10343718a43d5fcc669dd80"
I20260501 13:52:58.362756 1070 raft_consensus.cc:3060] T 9fc54df2e5e14019af12a8c0977f967f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Advancing to term 1
I20260501 13:52:58.363818 1464 tablet_bootstrap.cc:654] T 9fc54df2e5e14019af12a8c0977f967f P b0226350eae044398f04ef4e5cbb550e: Neither blocks nor log segments found. Creating new log.
W20260501 13:52:58.366256 1143 leader_election.cc:343] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Illegal state: must be running to vote when last-logged opid is not known
I20260501 13:52:58.367873 1464 tablet_bootstrap.cc:492] T 9fc54df2e5e14019af12a8c0977f967f P b0226350eae044398f04ef4e5cbb550e: No bootstrap required, opened a new log
I20260501 13:52:58.368041 1464 ts_tablet_manager.cc:1403] T 9fc54df2e5e14019af12a8c0977f967f P b0226350eae044398f04ef4e5cbb550e: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260501 13:52:58.368927 1464 raft_consensus.cc:359] T 9fc54df2e5e14019af12a8c0977f967f P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:58.369146 1464 raft_consensus.cc:385] T 9fc54df2e5e14019af12a8c0977f967f P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260501 13:52:58.369246 1464 raft_consensus.cc:740] T 9fc54df2e5e14019af12a8c0977f967f P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0226350eae044398f04ef4e5cbb550e, State: Initialized, Role: FOLLOWER
I20260501 13:52:58.369526 1464 consensus_queue.cc:260] T 9fc54df2e5e14019af12a8c0977f967f P b0226350eae044398f04ef4e5cbb550e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:58.370267 1464 ts_tablet_manager.cc:1434] T 9fc54df2e5e14019af12a8c0977f967f P b0226350eae044398f04ef4e5cbb550e: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260501 13:52:58.410208 721 catalog_manager.cc:5002] TS b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): tablet c6507fffd6a4466ab15b1eb89c7f32c7 (table default.table_5a0a455cca3c4ab6a059aac962b7cea0 [id=8818494b487941a4877edf5b079d9756]) successfully deleted
I20260501 13:52:58.436304 1209 raft_consensus.cc:2468] T 5c413244650e4784981922b6502f6b97 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0226350eae044398f04ef4e5cbb550e in term 1.
I20260501 13:52:58.436301 1071 raft_consensus.cc:2468] T 5c413244650e4784981922b6502f6b97 P e4a8a400d10343718a43d5fcc669dd80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0226350eae044398f04ef4e5cbb550e in term 1.
I20260501 13:52:58.437328 1279 leader_election.cc:304] T 5c413244650e4784981922b6502f6b97 P b0226350eae044398f04ef4e5cbb550e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e; no voters:
I20260501 13:52:58.437808 1469 raft_consensus.cc:2804] T 5c413244650e4784981922b6502f6b97 P b0226350eae044398f04ef4e5cbb550e [term 1 FOLLOWER]: Leader election won for term 1
I20260501 13:52:58.440200 1070 raft_consensus.cc:2468] T 9fc54df2e5e14019af12a8c0977f967f P e4a8a400d10343718a43d5fcc669dd80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1151db12ecae4dd7bb88b3d1415b7d92 in term 1.
I20260501 13:52:58.441133 1144 leader_election.cc:304] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1151db12ecae4dd7bb88b3d1415b7d92, e4a8a400d10343718a43d5fcc669dd80; no voters: b0226350eae044398f04ef4e5cbb550e
I20260501 13:52:58.441315 1481 ts_tablet_manager.cc:1929] T c6507fffd6a4466ab15b1eb89c7f32c7 P e4a8a400d10343718a43d5fcc669dd80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260501 13:52:58.441453 1481 log.cc:1199] T c6507fffd6a4466ab15b1eb89c7f32c7 P e4a8a400d10343718a43d5fcc669dd80: Deleting WAL directory at /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-0/wal/wals/c6507fffd6a4466ab15b1eb89c7f32c7
I20260501 13:52:58.441785 1493 raft_consensus.cc:2804] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92 [term 1 FOLLOWER]: Leader election won for term 1
I20260501 13:52:58.441931 1481 ts_tablet_manager.cc:1950] T c6507fffd6a4466ab15b1eb89c7f32c7 P e4a8a400d10343718a43d5fcc669dd80: Deleting consensus metadata
I20260501 13:52:58.441962 1493 raft_consensus.cc:697] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92 [term 1 LEADER]: Becoming Leader. State: Replica: 1151db12ecae4dd7bb88b3d1415b7d92, State: Running, Role: LEADER
I20260501 13:52:58.442338 1493 consensus_queue.cc:237] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:58.445569 1469 raft_consensus.cc:697] T 5c413244650e4784981922b6502f6b97 P b0226350eae044398f04ef4e5cbb550e [term 1 LEADER]: Becoming Leader. State: Replica: b0226350eae044398f04ef4e5cbb550e, State: Running, Role: LEADER
I20260501 13:52:58.446125 1469 consensus_queue.cc:237] T 5c413244650e4784981922b6502f6b97 P b0226350eae044398f04ef4e5cbb550e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
W20260501 13:52:58.461053 1475 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260501 13:52:58.461381 1475 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260501 13:52:58.461513 1475 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260501 13:52:58.470126 1475 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260501 13:52:58.470220 1475 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260501 13:52:58.470327 1475 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260501 13:52:58.470399 1475 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260501 13:52:58.481146 1475 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.105.20:38455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:33681
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.60:39729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.105.60
--webserver_port=43413
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision aa7cc0aa9c559b87f264235eaadd03ec492277e4
build type FASTDEBUG
built by None at 01 May 2026 13:43:23 UTC on bdcb31816ec0
build id 11694
ASAN enabled
I20260501 13:52:58.482877 1475 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260501 13:52:58.484949 1475 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260501 13:52:58.493330 1510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260501 13:52:58.493943 1507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260501 13:52:58.494475 1475 server_base.cc:1061] running on GCE node
W20260501 13:52:58.494721 1508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260501 13:52:58.495604 1475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260501 13:52:58.497118 1475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260501 13:52:58.498339 1475 hybrid_clock.cc:648] HybridClock initialized: now 1777643578498281 us; error 59 us; skew 500 ppm
I20260501 13:52:58.498700 1475 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260501 13:52:58.501058 1475 webserver.cc:492] Webserver started at http://127.0.105.60:43413/ using document root <none> and password file <none>
I20260501 13:52:58.501899 1475 fs_manager.cc:362] Metadata directory not provided
I20260501 13:52:58.502045 1475 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260501 13:52:58.507768 1475 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260501 13:52:58.510907 1516 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260501 13:52:58.512418 1475 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260501 13:52:58.512619 1475 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/data,/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/wal
uuid: "e47102ddbd4f41c1bf0a2864d97237b7"
format_stamp: "Formatted at 2026-05-01 13:52:54 on dist-test-slave-qlwv"
I20260501 13:52:58.513322 1475 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260501 13:52:58.529042 1475 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260501 13:52:58.529938 1475 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260501 13:52:58.530244 1475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260501 13:52:58.533509 722 catalog_manager.cc:5002] TS e4a8a400d10343718a43d5fcc669dd80 (127.0.105.1:44349): tablet c6507fffd6a4466ab15b1eb89c7f32c7 (table default.table_5a0a455cca3c4ab6a059aac962b7cea0 [id=8818494b487941a4877edf5b079d9756]) successfully deleted
I20260501 13:52:58.553061 1475 rpc_server.cc:307] RPC server started. Bound to: 127.0.105.60:39729
I20260501 13:52:58.553083 1567 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.105.60:39729 every 8 connection(s)
I20260501 13:52:58.554688 1475 server_base.cc:1193] Dumped server information to /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/data/info.pb
I20260501 13:52:58.557560 420 external_mini_cluster.cc:1428] Started /tmp/dist-test-task4ezAfy/build/asan/bin/kudu as pid 1475
I20260501 13:52:58.562623 1568 sys_catalog.cc:263] Verifying existing consensus state
I20260501 13:52:58.567504 1568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7: Bootstrap starting.
I20260501 13:52:58.576290 1568 log.cc:826] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7: Log is configured to *not* fsync() on all Append() calls
I20260501 13:52:58.584591 1568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260501 13:52:58.585062 1568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7: Bootstrap complete.
I20260501 13:52:58.590739 1568 raft_consensus.cc:359] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:52:58.591566 1568 raft_consensus.cc:740] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47102ddbd4f41c1bf0a2864d97237b7, State: Initialized, Role: FOLLOWER
I20260501 13:52:58.592145 1568 consensus_queue.cc:260] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:52:58.593070 1572 sys_catalog.cc:455] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:52:58.593365 1572 sys_catalog.cc:458] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [sys.catalog]: This master's current role is: FOLLOWER
I20260501 13:52:58.594136 1568 sys_catalog.cc:565] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260501 13:52:58.606942 1586 catalog_manager.cc:1269] Loaded cluster ID: 6424d308ce1d424b8fe7d2c1b812f4f7
I20260501 13:52:58.607787 1586 catalog_manager.cc:1562] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7: loading cluster ID for follower catalog manager: success
I20260501 13:52:58.616443 1586 catalog_manager.cc:1584] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7: acquiring CA information for follower catalog manager: success
I20260501 13:52:58.619762 1586 catalog_manager.cc:1612] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260501 13:52:58.642360 1591 raft_consensus.cc:493] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5fa23b4772a944c0b0279eabd7476196)
I20260501 13:52:58.642658 1591 raft_consensus.cc:515] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:52:58.643424 1591 leader_election.cc:290] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5fa23b4772a944c0b0279eabd7476196 (127.0.105.62:44705), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
I20260501 13:52:58.644134 746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "84a39a5f167243ff9bfe07189aa881ca" candidate_term: 3 candidate_status { last_received { term: 2 index: 17 } } ignore_live_leader: false dest_uuid: "5fa23b4772a944c0b0279eabd7476196" is_pre_election: true
I20260501 13:52:58.651187 1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "84a39a5f167243ff9bfe07189aa881ca" candidate_term: 3 candidate_status { last_received { term: 2 index: 17 } } ignore_live_leader: false dest_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" is_pre_election: true
I20260501 13:52:58.651855 1543 raft_consensus.cc:2468] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84a39a5f167243ff9bfe07189aa881ca in term 1.
I20260501 13:52:58.652375 790 leader_election.cc:304] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7; no voters: 5fa23b4772a944c0b0279eabd7476196
I20260501 13:52:58.652695 1591 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260501 13:52:58.652834 1591 raft_consensus.cc:493] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5fa23b4772a944c0b0279eabd7476196)
I20260501 13:52:58.652935 1591 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 2 FOLLOWER]: Advancing to term 3
I20260501 13:52:58.726996 1543 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 1 FOLLOWER]: Advancing to term 2
I20260501 13:52:58.742177 1591 raft_consensus.cc:515] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:52:58.744107 1591 leader_election.cc:290] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [CANDIDATE]: Term 3 election: Requested vote from peers 5fa23b4772a944c0b0279eabd7476196 (127.0.105.62:44705), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
I20260501 13:52:58.746546 746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "84a39a5f167243ff9bfe07189aa881ca" candidate_term: 3 candidate_status { last_received { term: 2 index: 17 } } ignore_live_leader: false dest_uuid: "5fa23b4772a944c0b0279eabd7476196"
I20260501 13:52:58.747330 1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "84a39a5f167243ff9bfe07189aa881ca" candidate_term: 3 candidate_status { last_received { term: 2 index: 17 } } ignore_live_leader: false dest_uuid: "e47102ddbd4f41c1bf0a2864d97237b7"
I20260501 13:52:58.748361 790 leader_election.cc:304] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84a39a5f167243ff9bfe07189aa881ca; no voters: 5fa23b4772a944c0b0279eabd7476196, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:52:58.749001 1591 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260501 13:52:58.769137 1572 sys_catalog.cc:455] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5fa23b4772a944c0b0279eabd7476196. Latest consensus state: current_term: 2 leader_uuid: "5fa23b4772a944c0b0279eabd7476196" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:52:58.769426 1572 sys_catalog.cc:458] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [sys.catalog]: This master's current role is: FOLLOWER
I20260501 13:52:58.769559 1596 sys_catalog.cc:455] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "5fa23b4772a944c0b0279eabd7476196" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:52:58.769897 1596 sys_catalog.cc:458] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [sys.catalog]: This master's current role is: FOLLOWER
I20260501 13:52:58.774554 1594 mvcc.cc:204] Tried to move back new op lower bound from 7281228096022065152 to 7281228092646334464. Current Snapshot: MvccSnapshot[applied={T|T < 7281228096022065152}]
I20260501 13:52:58.833267 1116 heartbeater.cc:344] Connected to a master server at 127.0.105.60:39729
I20260501 13:52:58.836483 1533 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" instance_seqno: 1777643575975284) as {username='slave'} at 127.0.105.1:42357; Asking this server to re-register.
I20260501 13:52:58.837420 1116 heartbeater.cc:461] Registering TS with master...
I20260501 13:52:58.839576 1533 ts_manager.cc:194] Registered new tserver with Master: e4a8a400d10343718a43d5fcc669dd80 (127.0.105.1:44349)
I20260501 13:52:58.957515 1388 heartbeater.cc:344] Connected to a master server at 127.0.105.60:39729
I20260501 13:52:58.957747 1388 heartbeater.cc:499] Master 127.0.105.60:39729 was elected leader, sending a full tablet report...
I20260501 13:52:58.957774 1254 heartbeater.cc:344] Connected to a master server at 127.0.105.60:39729
I20260501 13:52:58.960211 1533 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" instance_seqno: 1777643576999180) as {username='slave'} at 127.0.105.3:52615; Asking this server to re-register.
I20260501 13:52:58.960456 1532 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" instance_seqno: 1777643576483530) as {username='slave'} at 127.0.105.2:44617; Asking this server to re-register.
I20260501 13:52:58.961036 1388 heartbeater.cc:461] Registering TS with master...
I20260501 13:52:58.961062 1254 heartbeater.cc:461] Registering TS with master...
I20260501 13:52:58.962359 1532 ts_manager.cc:194] Registered new tserver with Master: 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
I20260501 13:52:58.962505 1533 ts_manager.cc:194] Registered new tserver with Master: b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841)
I20260501 13:52:59.013742 1603 raft_consensus.cc:493] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:52:59.013959 1603 raft_consensus.cc:515] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:59.014745 1603 leader_election.cc:290] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P b0226350eae044398f04ef4e5cbb550e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e4a8a400d10343718a43d5fcc669dd80 (127.0.105.1:44349), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
I20260501 13:52:59.015265 1070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "daa28e4cd0d042b5bb7f7ea04ddb19b7" candidate_uuid: "b0226350eae044398f04ef4e5cbb550e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4a8a400d10343718a43d5fcc669dd80" is_pre_election: true
I20260501 13:52:59.015326 1209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "daa28e4cd0d042b5bb7f7ea04ddb19b7" candidate_uuid: "b0226350eae044398f04ef4e5cbb550e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" is_pre_election: true
I20260501 13:52:59.015529 1209 raft_consensus.cc:2468] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0226350eae044398f04ef4e5cbb550e in term 0.
I20260501 13:52:59.015530 1070 raft_consensus.cc:2468] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0226350eae044398f04ef4e5cbb550e in term 0.
I20260501 13:52:59.016021 1279 leader_election.cc:304] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P b0226350eae044398f04ef4e5cbb550e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e; no voters:
I20260501 13:52:59.016332 1603 raft_consensus.cc:2804] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260501 13:52:59.016475 1603 raft_consensus.cc:493] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260501 13:52:59.016578 1603 raft_consensus.cc:3060] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Advancing to term 1
I20260501 13:52:59.041522 1603 raft_consensus.cc:515] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P b0226350eae044398f04ef4e5cbb550e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:59.042433 1603 leader_election.cc:290] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P b0226350eae044398f04ef4e5cbb550e [CANDIDATE]: Term 1 election: Requested vote from peers e4a8a400d10343718a43d5fcc669dd80 (127.0.105.1:44349), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
I20260501 13:52:59.043141 1070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "daa28e4cd0d042b5bb7f7ea04ddb19b7" candidate_uuid: "b0226350eae044398f04ef4e5cbb550e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4a8a400d10343718a43d5fcc669dd80"
I20260501 13:52:59.043439 1070 raft_consensus.cc:3060] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Advancing to term 1
I20260501 13:52:59.043947 1209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "daa28e4cd0d042b5bb7f7ea04ddb19b7" candidate_uuid: "b0226350eae044398f04ef4e5cbb550e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1151db12ecae4dd7bb88b3d1415b7d92"
I20260501 13:52:59.044143 1209 raft_consensus.cc:3060] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Advancing to term 1
I20260501 13:52:59.046147 1070 raft_consensus.cc:2468] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P e4a8a400d10343718a43d5fcc669dd80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0226350eae044398f04ef4e5cbb550e in term 1.
I20260501 13:52:59.046864 1278 leader_election.cc:304] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P b0226350eae044398f04ef4e5cbb550e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0226350eae044398f04ef4e5cbb550e, e4a8a400d10343718a43d5fcc669dd80; no voters:
I20260501 13:52:59.047355 728 catalog_manager.cc:2174] created HMS catalog entry for table [id=ac3b885b81d744bfaa69439967b2785f]
I20260501 13:52:59.047653 1603 raft_consensus.cc:2804] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P b0226350eae044398f04ef4e5cbb550e [term 1 FOLLOWER]: Leader election won for term 1
I20260501 13:52:59.048679 1603 raft_consensus.cc:697] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P b0226350eae044398f04ef4e5cbb550e [term 1 LEADER]: Becoming Leader. State: Replica: b0226350eae044398f04ef4e5cbb550e, State: Running, Role: LEADER
I20260501 13:52:59.049003 1603 consensus_queue.cc:237] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P b0226350eae044398f04ef4e5cbb550e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:59.067051 1456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "5fa23b4772a944c0b0279eabd7476196" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:52:59.067835 1456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: This master's current role is: LEADER
I20260501 13:52:59.069993 815 raft_consensus.cc:1240] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 3 FOLLOWER]: Rejecting Update request from peer 5fa23b4772a944c0b0279eabd7476196 for earlier term 2. Current term is 3. Ops: [2.18-2.18]
I20260501 13:52:59.077884 1607 consensus_queue.cc:1059] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 }, Status: INVALID_TERM, Last received: 2.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260501 13:52:59.081406 1611 raft_consensus.cc:3055] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 2 LEADER]: Stepping down as leader of term 2
I20260501 13:52:59.081612 1611 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5fa23b4772a944c0b0279eabd7476196, State: Running, Role: LEADER
I20260501 13:52:59.082448 728 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35974 (request call id 15) took 1661 ms. Trace:
I20260501 13:52:59.082631 1611 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 2.18, Last appended by leader: 18, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:52:59.083702 815 raft_consensus.cc:1240] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 3 FOLLOWER]: Rejecting Update request from peer 5fa23b4772a944c0b0279eabd7476196 for earlier term 2. Current term is 3. Ops: [2.18-2.18]
I20260501 13:52:59.084339 1613 consensus_queue.cc:1048] T 3cb76051f80b49439f8b9191af3cef97 P 1151db12ecae4dd7bb88b3d1415b7d92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260501 13:52:59.096490 1618 consensus_queue.cc:1048] T 3cb76051f80b49439f8b9191af3cef97 P 1151db12ecae4dd7bb88b3d1415b7d92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260501 13:52:59.111725 1611 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 2 FOLLOWER]: Advancing to term 3
I20260501 13:52:59.082880 728 rpcz_store.cc:276] 0501 13:52:57.420496 (+ 0us) service_pool.cc:167] Inserting onto call queue
0501 13:52:57.422173 (+ 1677us) service_pool.cc:224] Handling call
0501 13:52:57.678686 (+256513us) catalog_manager.cc:2101] Acquired catalog manager lock
0501 13:52:57.679102 (+ 416us) catalog_manager.cc:2147] Created new table and tablet info
0501 13:52:59.047306 (+1368204us) catalog_manager.cc:2173] Created new table in HMS catalog
0501 13:52:59.082105 (+ 34799us) catalog_manager.cc:2207] Wrote table and tablets to system table
0501 13:52:59.082291 (+ 186us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0501 13:52:59.082402 (+ 111us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0501 13:52:59.048711 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0501 13:52:59.048962 (+ 251us) write_op.cc:432] Acquiring schema lock in shared mode
0501 13:52:59.048971 (+ 9us) write_op.cc:435] Acquired schema lock
0501 13:52:59.048975 (+ 4us) tablet.cc:662] Decoding operations
0501 13:52:59.049080 (+ 105us) write_op.cc:620] Acquiring the partition lock for write op
0501 13:52:59.049095 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0501 13:52:59.049100 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0501 13:52:59.049274 (+ 174us) tablet.cc:701] Row locks acquired
0501 13:52:59.049277 (+ 3us) write_op.cc:260] PREPARE: finished
0501 13:52:59.049315 (+ 38us) write_op.cc:270] Start()
0501 13:52:59.049350 (+ 35us) write_op.cc:276] Timestamp: P: 1777643579049280 usec, L: 0
0501 13:52:59.049353 (+ 3us) op_driver.cc:348] REPLICATION: starting
0501 13:52:59.049571 (+ 218us) log.cc:844] Serialized 432 byte log entry
0501 13:52:59.065125 (+ 15554us) spinlock_profiling.cc:243] Waited 6.83 ms on lock 0x6160000421b8. stack: 00007f90b3c89086 00007f90b3c88e40 00007f90b2c423ad 000000000099adb4 000000000099ad7d 000000000099a74a 00007f90bfac53f9 00007f90bfaca7e4 00007f90bfaca5a8 0000000000aa4cac 00007f90b3ccef6f 00007f90b3cd228b 00007f90b3cd2108 0000000000aa4cac 00007f90b3cae528 00007f90b727c6da
0501 13:52:59.068660 (+ 3535us) spinlock_profiling.cc:243] Waited 3.72 ms on lock 0x6160000421b8. stack: 00007f90b3c89086 00007f90b3c88e40 00007f90b2c423ad 000000000099adb4 000000000099ad7d 000000000099a74a 00007f90bfab6f2f 00007f90bfa84c3b 00007f90bfa8fbd1 00007f90bfa8f8bd 0000000000aa4cac 00007f90b3ccef6f 00007f90b3cd228b 00007f90b3cd2108 0000000000aa4cac 00007f90b3cae528
0501 13:52:59.078942 (+ 10282us) op_driver.cc:464] REPLICATION: finished
0501 13:52:59.081296 (+ 2354us) write_op.cc:301] APPLY: starting
0501 13:52:59.081314 (+ 18us) tablet.cc:1367] starting BulkCheckPresence
0501 13:52:59.081366 (+ 52us) tablet.cc:1370] finished BulkCheckPresence
0501 13:52:59.081368 (+ 2us) tablet.cc:1372] starting ApplyRowOperation cycle
0501 13:52:59.081772 (+ 404us) tablet.cc:1383] finished ApplyRowOperation cycle
0501 13:52:59.081806 (+ 34us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0501 13:52:59.081814 (+ 8us) write_op.cc:312] APPLY: finished
0501 13:52:59.081912 (+ 98us) log.cc:844] Serialized 28 byte log entry
0501 13:52:59.081991 (+ 79us) write_op.cc:489] Releasing partition, row and schema locks
0501 13:52:59.082013 (+ 22us) write_op.cc:454] Released schema lock
0501 13:52:59.082020 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0501 13:52:59.082099 (+ 79us) write_op.cc:489] Releasing partition, row and schema locks
0501 13:52:59.082103 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":412407,"Hive Metastore.run_cpu_time_us":296,"Hive Metastore.run_wall_time_us":955443,"child_traces":[["op",{"apply.queue_time_us":2263,"apply.run_cpu_time_us":775,"apply.run_wall_time_us":805,"leader-initialization.queue_time_us":9409,"leader-initialization.run_cpu_time_us":27,"leader-initialization.run_wall_time_us":28,"num_ops":2,"prepare.queue_time_us":407,"prepare.run_cpu_time_us":1760,"prepare.run_wall_time_us":9245,"raft.queue_time_us":18066,"raft.run_cpu_time_us":22007,"raft.run_wall_time_us":60589,"replication_time_us":29550,"spinlock_wait_cycles":24269440,"thread_start_us":42012,"threads_started":6}]]}
I20260501 13:52:59.141043 1209 raft_consensus.cc:2468] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0226350eae044398f04ef4e5cbb550e in term 1.
W20260501 13:52:59.141737 728 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.105.3:54865 (request call id 7) took 996 ms (client timeout 1000 ms). Trace:
W20260501 13:52:59.141881 728 rpcz_store.cc:269] 0501 13:52:58.145124 (+ 0us) service_pool.cc:167] Inserting onto call queue
0501 13:52:59.112512 (+967388us) service_pool.cc:224] Handling call
0501 13:52:59.141291 (+ 28779us) spinlock_profiling.cc:243] Waited 28.3 ms on lock 0x616000041254. stack: 00007f90b3c89086 00007f90b3c88e40 00007f90b2c423ad 000000000099adb4 000000000099ad7d 00007f90c2ff16bd 00007f90c2ff17b3 00007f90bfbafb21 00007f90c78e3bad 00007f90c7a6a358 00007f90be500ca6 00007f90be5009d5 00007f90bac5c10e 00007f90bac5afee 00007f90bac61228 00007f90bac61b8f
0501 13:52:59.141684 (+ 393us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":65103744}
W20260501 13:52:59.142740 728 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.105.1:57247 (request call id 9) took 996 ms (client timeout 1000 ms). Trace:
W20260501 13:52:59.142875 728 rpcz_store.cc:269] 0501 13:52:58.146298 (+ 0us) service_pool.cc:167] Inserting onto call queue
0501 13:52:59.142225 (+995927us) service_pool.cc:224] Handling call
0501 13:52:59.142701 (+ 476us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260501 13:52:59.143532 728 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.105.2:34613 (request call id 10) took 924 ms (client timeout 1000 ms). Trace:
W20260501 13:52:59.143692 728 rpcz_store.cc:269] 0501 13:52:58.218799 (+ 0us) service_pool.cc:167] Inserting onto call queue
0501 13:52:59.143058 (+924259us) service_pool.cc:224] Handling call
0501 13:52:59.143500 (+ 442us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260501 13:52:59.148474 728 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35974 (request call id 26) took 1157 ms. Trace:
I20260501 13:52:59.148664 728 rpcz_store.cc:276] 0501 13:52:57.991384 (+ 0us) service_pool.cc:167] Inserting onto call queue
0501 13:52:59.147964 (+1156580us) service_pool.cc:224] Handling call
0501 13:52:59.148434 (+ 470us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260501 13:52:59.149168 733 catalog_manager.cc:2174] created HMS catalog entry for table [id=6a33ac064eb049ba8e11ce019c07d436]
I20260501 13:52:59.149689 728 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:35974 (request call id 27) took 1033 ms. Trace:
W20260501 13:52:59.149755 1398 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.105.62:44705)
I20260501 13:52:59.149798 728 rpcz_store.cc:276] 0501 13:52:58.116392 (+ 0us) service_pool.cc:167] Inserting onto call queue
0501 13:52:59.149341 (+1032949us) service_pool.cc:224] Handling call
0501 13:52:59.149639 (+ 298us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260501 13:52:59.150187 733 catalog_manager.cc:2202] Illegal state: an error occurred while writing to the sys-catalog: replica 5fa23b4772a944c0b0279eabd7476196 is not leader of this config: current role FOLLOWER
I20260501 13:52:59.152230 728 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35974 (request call id 28) took 1016 ms. Trace:
I20260501 13:52:59.152390 728 rpcz_store.cc:276] 0501 13:52:58.135776 (+ 0us) service_pool.cc:167] Inserting onto call queue
0501 13:52:59.151931 (+1016155us) service_pool.cc:224] Handling call
0501 13:52:59.152199 (+ 268us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260501 13:52:59.152760 728 master_service.cc:954] ReplaceTablet: received request to replace tablet 5c413244650e4784981922b6502f6b97 from {username='slave'} at 127.0.0.1:35974
I20260501 13:52:59.153848 728 master_service.cc:954] ReplaceTablet: received request to replace tablet 3cb76051f80b49439f8b9191af3cef97 from {username='slave'} at 127.0.0.1:35974
I20260501 13:52:59.154349 728 master_service.cc:954] ReplaceTablet: received request to replace tablet 5c413244650e4784981922b6502f6b97 from {username='slave'} at 127.0.0.1:35974
I20260501 13:52:59.154882 728 master_service.cc:954] ReplaceTablet: received request to replace tablet c6507fffd6a4466ab15b1eb89c7f32c7 from {username='slave'} at 127.0.0.1:35974
I20260501 13:52:59.155396 728 master_service.cc:954] ReplaceTablet: received request to replace tablet 9fc54df2e5e14019af12a8c0977f967f from {username='slave'} at 127.0.0.1:35974
W20260501 13:52:59.158831 1398 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.105.62:44705)
W20260501 13:52:59.159824 1398 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.105.62:44705)
I20260501 13:52:59.188908 1475 master_runner.cc:263] No leader master found from master e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:52:59.189090 1475 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:52:59.276993 1342 raft_consensus.cc:3060] T 9fc54df2e5e14019af12a8c0977f967f P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Advancing to term 1
I20260501 13:52:59.303478 1625 raft_consensus.cc:493] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:52:59.303714 1625 raft_consensus.cc:515] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:52:59.304657 1625 leader_election.cc:290] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5fa23b4772a944c0b0279eabd7476196 (127.0.105.62:44705), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
I20260501 13:52:59.305007 746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "84a39a5f167243ff9bfe07189aa881ca" candidate_term: 4 candidate_status { last_received { term: 2 index: 17 } } ignore_live_leader: false dest_uuid: "5fa23b4772a944c0b0279eabd7476196" is_pre_election: true
I20260501 13:52:59.305321 746 raft_consensus.cc:2410] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84a39a5f167243ff9bfe07189aa881ca for term 4 because replica has last-logged OpId of term: 2 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 17.
I20260501 13:52:59.305230 1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "84a39a5f167243ff9bfe07189aa881ca" candidate_term: 4 candidate_status { last_received { term: 2 index: 17 } } ignore_live_leader: false dest_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" is_pre_election: true
I20260501 13:52:59.305938 790 leader_election.cc:304] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84a39a5f167243ff9bfe07189aa881ca; no voters: 5fa23b4772a944c0b0279eabd7476196, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:52:59.306232 1625 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260501 13:52:59.346733 729 catalog_manager.cc:2174] created HMS catalog entry for table [id=d1738ac2ccf94e2bbcf36e03630922fc]
W20260501 13:52:59.347813 729 catalog_manager.cc:2202] Illegal state: an error occurred while writing to the sys-catalog: replica 5fa23b4772a944c0b0279eabd7476196 is not leader of this config: current role FOLLOWER
I20260501 13:52:59.352650 1618 consensus_queue.cc:1048] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260501 13:52:59.353709 1603 raft_consensus.cc:493] T 9fc54df2e5e14019af12a8c0977f967f P b0226350eae044398f04ef4e5cbb550e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:52:59.353897 1603 raft_consensus.cc:515] T 9fc54df2e5e14019af12a8c0977f967f P b0226350eae044398f04ef4e5cbb550e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:59.354593 1603 leader_election.cc:290] T 9fc54df2e5e14019af12a8c0977f967f P b0226350eae044398f04ef4e5cbb550e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e4a8a400d10343718a43d5fcc669dd80 (127.0.105.1:44349), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
I20260501 13:52:59.355129 1070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc54df2e5e14019af12a8c0977f967f" candidate_uuid: "b0226350eae044398f04ef4e5cbb550e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4a8a400d10343718a43d5fcc669dd80" is_pre_election: true
I20260501 13:52:59.355358 1070 raft_consensus.cc:2468] T 9fc54df2e5e14019af12a8c0977f967f P e4a8a400d10343718a43d5fcc669dd80 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0226350eae044398f04ef4e5cbb550e in term 1.
I20260501 13:52:59.355736 1209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc54df2e5e14019af12a8c0977f967f" candidate_uuid: "b0226350eae044398f04ef4e5cbb550e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" is_pre_election: true
I20260501 13:52:59.355815 1278 leader_election.cc:304] T 9fc54df2e5e14019af12a8c0977f967f P b0226350eae044398f04ef4e5cbb550e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0226350eae044398f04ef4e5cbb550e, e4a8a400d10343718a43d5fcc669dd80; no voters:
I20260501 13:52:59.356081 1603 raft_consensus.cc:2804] T 9fc54df2e5e14019af12a8c0977f967f P b0226350eae044398f04ef4e5cbb550e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260501 13:52:59.356166 1603 raft_consensus.cc:493] T 9fc54df2e5e14019af12a8c0977f967f P b0226350eae044398f04ef4e5cbb550e [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260501 13:52:59.356259 1603 raft_consensus.cc:3060] T 9fc54df2e5e14019af12a8c0977f967f P b0226350eae044398f04ef4e5cbb550e [term 1 FOLLOWER]: Advancing to term 2
I20260501 13:52:59.454895 1603 raft_consensus.cc:515] T 9fc54df2e5e14019af12a8c0977f967f P b0226350eae044398f04ef4e5cbb550e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:59.455408 1342 raft_consensus.cc:1240] T 9fc54df2e5e14019af12a8c0977f967f P b0226350eae044398f04ef4e5cbb550e [term 2 FOLLOWER]: Rejecting Update request from peer 1151db12ecae4dd7bb88b3d1415b7d92 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260501 13:52:59.455729 1603 leader_election.cc:290] T 9fc54df2e5e14019af12a8c0977f967f P b0226350eae044398f04ef4e5cbb550e [CANDIDATE]: Term 2 election: Requested vote from peers e4a8a400d10343718a43d5fcc669dd80 (127.0.105.1:44349), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
I20260501 13:52:59.456624 1209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc54df2e5e14019af12a8c0977f967f" candidate_uuid: "b0226350eae044398f04ef4e5cbb550e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1151db12ecae4dd7bb88b3d1415b7d92"
I20260501 13:52:59.457010 1070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc54df2e5e14019af12a8c0977f967f" candidate_uuid: "b0226350eae044398f04ef4e5cbb550e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4a8a400d10343718a43d5fcc669dd80"
I20260501 13:52:59.457211 1070 raft_consensus.cc:3060] T 9fc54df2e5e14019af12a8c0977f967f P e4a8a400d10343718a43d5fcc669dd80 [term 1 FOLLOWER]: Advancing to term 2
I20260501 13:52:59.457350 1618 consensus_queue.cc:1059] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260501 13:52:59.457630 1618 raft_consensus.cc:3055] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92 [term 1 LEADER]: Stepping down as leader of term 1
I20260501 13:52:59.457751 1618 raft_consensus.cc:740] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1151db12ecae4dd7bb88b3d1415b7d92, State: Running, Role: LEADER
I20260501 13:52:59.458050 1618 consensus_queue.cc:260] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:59.458552 1618 raft_consensus.cc:3060] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92 [term 1 FOLLOWER]: Advancing to term 2
I20260501 13:52:59.487646 1603 consensus_queue.cc:1048] T 5c413244650e4784981922b6502f6b97 P b0226350eae044398f04ef4e5cbb550e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260501 13:52:59.497993 1603 consensus_queue.cc:1048] T 5c413244650e4784981922b6502f6b97 P b0226350eae044398f04ef4e5cbb550e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260501 13:52:59.511703 1611 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:52:59.511899 1611 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:52:59.512619 1611 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
I20260501 13:52:59.513325 815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5fa23b4772a944c0b0279eabd7476196" candidate_term: 4 candidate_status { last_received { term: 2 index: 18 } } ignore_live_leader: false dest_uuid: "84a39a5f167243ff9bfe07189aa881ca" is_pre_election: true
I20260501 13:52:59.513582 815 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fa23b4772a944c0b0279eabd7476196 in term 3.
I20260501 13:52:59.513726 1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5fa23b4772a944c0b0279eabd7476196" candidate_term: 4 candidate_status { last_received { term: 2 index: 18 } } ignore_live_leader: false dest_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" is_pre_election: true
I20260501 13:52:59.514163 723 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196, 84a39a5f167243ff9bfe07189aa881ca; no voters:
I20260501 13:52:59.514711 1611 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260501 13:52:59.514839 1611 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260501 13:52:59.514940 1611 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 3 FOLLOWER]: Advancing to term 4
I20260501 13:52:59.528621 1070 raft_consensus.cc:2468] T 9fc54df2e5e14019af12a8c0977f967f P e4a8a400d10343718a43d5fcc669dd80 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0226350eae044398f04ef4e5cbb550e in term 2.
I20260501 13:52:59.529455 1278 leader_election.cc:304] T 9fc54df2e5e14019af12a8c0977f967f P b0226350eae044398f04ef4e5cbb550e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b0226350eae044398f04ef4e5cbb550e, e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92
I20260501 13:52:59.529613 1611 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:52:59.532801 732 catalog_manager.cc:2174] created HMS catalog entry for table [id=a7f494192d5145a2b9c3c7c836b8dd9f]
W20260501 13:52:59.533931 732 catalog_manager.cc:2202] Illegal state: an error occurred while writing to the sys-catalog: replica 5fa23b4772a944c0b0279eabd7476196 is not leader of this config: current role FOLLOWER
I20260501 13:52:59.538990 1611 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 4 election: Requested vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
I20260501 13:52:59.539402 815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5fa23b4772a944c0b0279eabd7476196" candidate_term: 4 candidate_status { last_received { term: 2 index: 18 } } ignore_live_leader: false dest_uuid: "84a39a5f167243ff9bfe07189aa881ca"
I20260501 13:52:59.539552 1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5fa23b4772a944c0b0279eabd7476196" candidate_term: 4 candidate_status { last_received { term: 2 index: 18 } } ignore_live_leader: false dest_uuid: "e47102ddbd4f41c1bf0a2864d97237b7"
I20260501 13:52:59.539673 815 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 3 FOLLOWER]: Advancing to term 4
I20260501 13:52:59.538702 1603 raft_consensus.cc:2804] T 9fc54df2e5e14019af12a8c0977f967f P b0226350eae044398f04ef4e5cbb550e [term 2 FOLLOWER]: Leader election won for term 2
I20260501 13:52:59.540352 1603 raft_consensus.cc:697] T 9fc54df2e5e14019af12a8c0977f967f P b0226350eae044398f04ef4e5cbb550e [term 2 LEADER]: Becoming Leader. State: Replica: b0226350eae044398f04ef4e5cbb550e, State: Running, Role: LEADER
I20260501 13:52:59.540652 1603 consensus_queue.cc:237] T 9fc54df2e5e14019af12a8c0977f967f P b0226350eae044398f04ef4e5cbb550e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:52:59.589835 1632 raft_consensus.cc:493] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5fa23b4772a944c0b0279eabd7476196)
I20260501 13:52:59.590070 1632 raft_consensus.cc:515] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:52:59.592875 746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 18 } } ignore_live_leader: false dest_uuid: "5fa23b4772a944c0b0279eabd7476196" is_pre_election: true
I20260501 13:52:59.593195 746 raft_consensus.cc:2368] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e47102ddbd4f41c1bf0a2864d97237b7 for earlier term 3. Current term is 4.
I20260501 13:52:59.593317 1632 leader_election.cc:290] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5fa23b4772a944c0b0279eabd7476196 (127.0.105.62:44705), 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209)
I20260501 13:52:59.593899 1517 leader_election.cc:400] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 5fa23b4772a944c0b0279eabd7476196 (127.0.105.62:44705) with higher term. Message: Invalid argument: T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e47102ddbd4f41c1bf0a2864d97237b7 for earlier term 3. Current term is 4.
I20260501 13:52:59.593990 1517 leader_election.cc:403] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260501 13:52:59.594468 814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 18 } } ignore_live_leader: false dest_uuid: "84a39a5f167243ff9bfe07189aa881ca" is_pre_election: true
I20260501 13:52:59.594908 1632 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 2 FOLLOWER]: Advancing to term 4
I20260501 13:52:59.628585 815 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fa23b4772a944c0b0279eabd7476196 in term 4.
I20260501 13:52:59.628809 1632 raft_consensus.cc:2749] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 5fa23b4772a944c0b0279eabd7476196 (127.0.105.62:44705) with higher term. Message: Invalid argument: T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e47102ddbd4f41c1bf0a2864d97237b7 for earlier term 3. Current term is 4.
I20260501 13:52:59.629400 723 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196, 84a39a5f167243ff9bfe07189aa881ca; no voters: e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:52:59.630630 1611 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 4 FOLLOWER]: Leader election won for term 4
I20260501 13:52:59.634418 1611 raft_consensus.cc:697] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 4 LEADER]: Becoming Leader. State: Replica: 5fa23b4772a944c0b0279eabd7476196, State: Running, Role: LEADER
I20260501 13:52:59.634812 1611 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 2.18, Last appended by leader: 18, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:52:59.636746 1633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5fa23b4772a944c0b0279eabd7476196. Latest consensus state: current_term: 4 leader_uuid: "5fa23b4772a944c0b0279eabd7476196" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:52:59.637071 1633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: This master's current role is: LEADER
13:52:59.781 [HMS - ERROR - pool-6-thread-1] (RetryingHMSHandler.java:201) AlreadyExistsException(message:Table hive.default.table_86cdd461d6044be8b0ae522dfcc0714a already exists)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1839)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1786)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_with_environment_context(HiveMetaStore.java:2035)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:569)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
at jdk.proxy2/jdk.proxy2.$Proxy27.create_table_with_environment_context(Unknown Source)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:14800)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:14784)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:104)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
W20260501 13:52:59.783695 731 catalog_manager.cc:2170] Already present: failed to create HMS catalog entry for table [id=0701fd43ae29412e87a55c0db0713223]: failed to create Hive MetaStore table: TException - service has thrown: AlreadyExistsException(message=Table hive.default.table_86cdd461d6044be8b0ae522dfcc0714a already exists)
I20260501 13:52:59.784161 731 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35974 (request call id 12) took 2363 ms. Trace:
I20260501 13:52:59.784287 731 rpcz_store.cc:276] 0501 13:52:57.420225 (+ 0us) service_pool.cc:167] Inserting onto call queue
0501 13:52:57.421397 (+ 1172us) service_pool.cc:224] Handling call
0501 13:52:57.682318 (+260921us) catalog_manager.cc:2101] Acquired catalog manager lock
0501 13:52:57.682727 (+ 409us) catalog_manager.cc:2147] Created new table and tablet info
0501 13:52:59.784122 (+2101395us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":1849585,"Hive Metastore.run_cpu_time_us":981,"Hive Metastore.run_wall_time_us":251094}
I20260501 13:52:59.821691 1603 consensus_queue.cc:1048] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P b0226350eae044398f04ef4e5cbb550e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260501 13:52:59.832393 1628 consensus_queue.cc:1048] T daa28e4cd0d042b5bb7f7ea04ddb19b7 P b0226350eae044398f04ef4e5cbb550e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260501 13:52:59.878124 734 catalog_manager.cc:2174] created HMS catalog entry for table [id=b0561cbad3a94637a54b01f351d94fbb]
I20260501 13:52:59.883942 815 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 4 FOLLOWER]: Refusing update from remote peer 5fa23b4772a944c0b0279eabd7476196: Log matching property violated. Preceding OpId in replica: term: 2 index: 17. Preceding OpId from leader: term: 4 index: 20. (index mismatch)
I20260501 13:52:59.883958 1543 raft_consensus.cc:1275] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 4 FOLLOWER]: Refusing update from remote peer 5fa23b4772a944c0b0279eabd7476196: Log matching property violated. Preceding OpId in replica: term: 2 index: 18. Preceding OpId from leader: term: 4 index: 20. (index mismatch)
I20260501 13:52:59.884666 1633 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.000s
I20260501 13:52:59.886153 1611 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
F20260501 13:52:59.893127 936 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 10 received events: NotificationEvent(eventId=1, eventTime=1777643577, eventType=CREATE_TABLE, dbName=default, tableName=table_5a0a455cca3c4ab6a059aac962b7cea0, message={"server":"","servicePrincipal":"","db":"default","table":"table_5a0a455cca3c4ab6a059aac962b7cea0","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_5a0a455cca3c4ab6a059aac962b7cea0\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1777643577},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/warehouse/table_5a0a455cca3c4ab6a059aac962b7cea0\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_5a0a455cca3c4ab6a059aac962b7cea0\",\"kudu.table_id\":\"8818494b487941a4877edf5b079d9756\",\"transient_lastDdlTime\":\"1777643577\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"6424d308ce1d424b8fe7d2c1b812f4f7\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1777643577,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1777643577, eventType=CREATE_TABLE, dbName=default, tableName=table_86cdd461d6044be8b0ae522dfcc0714a, message={"server":"","servicePrincipal":"","db":"default","table":"table_86cdd461d6044be8b0ae522dfcc0714a","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_86cdd461d6044be8b0ae522dfcc0714a\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1777643577},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/warehouse/table_86cdd461d6044be8b0ae522dfcc0714a\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_86cdd461d6044be8b0ae522dfcc0714a\",\"kudu.table_id\":\"df2e28d52fd341ffac31641fe0482fe8\",\"transient_lastDdlTime\":\"1777643577\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"6424d308ce1d424b8fe7d2c1b812f4f7\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1777643577,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
@ 0x7f90b0f15dcd google::LogMessage::Fail() at ??:0
I20260501 13:52:59.899436 1632 sys_catalog.cc:455] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5fa23b4772a944c0b0279eabd7476196. Latest consensus state: current_term: 4 leader_uuid: "5fa23b4772a944c0b0279eabd7476196" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:52:59.899691 1632 sys_catalog.cc:458] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [sys.catalog]: This master's current role is: FOLLOWER
@ 0x7f90b0f19b93 google::LogMessage::SendToLog() at ??:0
@ 0x7f90b0f157cc google::LogMessage::Flush() at ??:0
@ 0x7f90b0f15a11 google::LogMessage::~LogMessage() at ??:0
I20260501 13:52:59.918756 1633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "5fa23b4772a944c0b0279eabd7476196" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:52:59.920022 1633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: This master's current role is: LEADER
I20260501 13:52:59.920203 734 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35974 (request call id 9) took 2499 ms. Trace:
I20260501 13:52:59.920317 734 rpcz_store.cc:276] 0501 13:52:57.419951 (+ 0us) service_pool.cc:167] Inserting onto call queue
0501 13:52:57.420022 (+ 71us) service_pool.cc:224] Handling call
0501 13:52:57.683001 (+262979us) catalog_manager.cc:2101] Acquired catalog manager lock
0501 13:52:57.683444 (+ 443us) catalog_manager.cc:2147] Created new table and tablet info
0501 13:52:59.878050 (+2194606us) catalog_manager.cc:2173] Created new table in HMS catalog
0501 13:52:59.919355 (+ 41305us) catalog_manager.cc:2207] Wrote table and tablets to system table
0501 13:52:59.919477 (+ 122us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0501 13:52:59.919639 (+ 162us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0501 13:52:59.881715 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0501 13:52:59.881887 (+ 172us) write_op.cc:432] Acquiring schema lock in shared mode
0501 13:52:59.881894 (+ 7us) write_op.cc:435] Acquired schema lock
0501 13:52:59.881897 (+ 3us) tablet.cc:662] Decoding operations
0501 13:52:59.881999 (+ 102us) write_op.cc:620] Acquiring the partition lock for write op
0501 13:52:59.882014 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0501 13:52:59.882019 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0501 13:52:59.882102 (+ 83us) tablet.cc:701] Row locks acquired
0501 13:52:59.882105 (+ 3us) write_op.cc:260] PREPARE: finished
0501 13:52:59.882166 (+ 61us) write_op.cc:270] Start()
0501 13:52:59.882204 (+ 38us) write_op.cc:276] Timestamp: P: 1777643579882106 usec, L: 0
0501 13:52:59.882207 (+ 3us) op_driver.cc:348] REPLICATION: starting
0501 13:52:59.882445 (+ 238us) log.cc:844] Serialized 432 byte log entry
0501 13:52:59.898720 (+ 16275us) op_driver.cc:464] REPLICATION: finished
0501 13:52:59.918535 (+ 19815us) write_op.cc:301] APPLY: starting
0501 13:52:59.918559 (+ 24us) tablet.cc:1367] starting BulkCheckPresence
0501 13:52:59.918610 (+ 51us) tablet.cc:1370] finished BulkCheckPresence
0501 13:52:59.918666 (+ 56us) tablet.cc:1372] starting ApplyRowOperation cycle
0501 13:52:59.918948 (+ 282us) tablet.cc:1383] finished ApplyRowOperation cycle
0501 13:52:59.918991 (+ 43us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0501 13:52:59.918999 (+ 8us) write_op.cc:312] APPLY: finished
0501 13:52:59.919108 (+ 109us) log.cc:844] Serialized 28 byte log entry
0501 13:52:59.919203 (+ 95us) write_op.cc:489] Releasing partition, row and schema locks
0501 13:52:59.919252 (+ 49us) write_op.cc:454] Released schema lock
0501 13:52:59.919259 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0501 13:52:59.919743 (+ 484us) write_op.cc:489] Releasing partition, row and schema locks
0501 13:52:59.919749 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":2100011,"Hive Metastore.run_cpu_time_us":381,"Hive Metastore.run_wall_time_us":94408,"spinlock_wait_cycles":6400,"child_traces":[["op",{"apply.queue_time_us":19724,"apply.run_cpu_time_us":794,"apply.run_wall_time_us":1212,"num_ops":2,"prepare.queue_time_us":2735,"prepare.run_cpu_time_us":899,"prepare.run_wall_time_us":900,"raft.queue_time_us":473,"raft.run_cpu_time_us":397,"raft.run_wall_time_us":394,"replication_time_us":16479,"thread_start_us":22170,"threads_started":2}]]}
W20260501 13:52:59.924185 1398 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.0.105.62:44705)
@ 0x7f90c79ce574 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260501 13:52:59.929129 1632 sys_catalog.cc:455] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "5fa23b4772a944c0b0279eabd7476196" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:52:59.929406 1632 sys_catalog.cc:458] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [sys.catalog]: This master's current role is: FOLLOWER
@ 0x7f90c79cce21 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260501 13:52:59.942742 1638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5fa23b4772a944c0b0279eabd7476196. Latest consensus state: current_term: 4 leader_uuid: "5fa23b4772a944c0b0279eabd7476196" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:52:59.943017 1638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [sys.catalog]: This master's current role is: FOLLOWER
@ 0x7f90c79d4a9c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260501 13:52:59.945652 1633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "5fa23b4772a944c0b0279eabd7476196" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:52:59.945957 1633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: This master's current role is: LEADER
I20260501 13:52:59.947348 1638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "5fa23b4772a944c0b0279eabd7476196" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:52:59.947566 1638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [sys.catalog]: This master's current role is: FOLLOWER
@ 0x7f90c79d4919 std::_Function_handler<>::_M_invoke() at ??:0
@ 0xaa4cad std::function<>::operator()() at ??:?
@ 0x7f90b3cae529 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f90b727c6db start_thread at ??:0
@ 0x7f90afdc661f clone at ??:0
W20260501 13:53:00.333669 1142 connection.cc:570] client connection to 127.0.105.62:44705 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260501 13:53:00.333767 1398 connection.cc:570] client connection to 127.0.105.62:44705 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260501 13:53:00.334153 1398 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.105.62:44705)
W20260501 13:53:00.334496 1276 connection.cc:570] client connection to 127.0.105.62:44705 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260501 13:53:00.334856 1398 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.105.62:44705)
I20260501 13:53:00.400096 1632 raft_consensus.cc:493] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5fa23b4772a944c0b0279eabd7476196)
I20260501 13:53:00.400324 1632 raft_consensus.cc:515] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:00.401026 1632 leader_election.cc:290] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5fa23b4772a944c0b0279eabd7476196 (127.0.105.62:44705), 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209)
I20260501 13:53:00.401687 815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 20 } } ignore_live_leader: false dest_uuid: "84a39a5f167243ff9bfe07189aa881ca" is_pre_election: true
W20260501 13:53:00.402372 1517 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.62:44705: connect: Connection refused (error 111)
W20260501 13:53:00.403822 1517 leader_election.cc:336] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5fa23b4772a944c0b0279eabd7476196 (127.0.105.62:44705): Network error: Client connection negotiation failed: client connection to 127.0.105.62:44705: connect: Connection refused (error 111)
I20260501 13:53:00.403947 1517 leader_election.cc:304] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e47102ddbd4f41c1bf0a2864d97237b7; no voters: 5fa23b4772a944c0b0279eabd7476196, 84a39a5f167243ff9bfe07189aa881ca
I20260501 13:53:00.404299 1632 raft_consensus.cc:2749] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260501 13:53:00.448583 1665 raft_consensus.cc:493] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5fa23b4772a944c0b0279eabd7476196)
I20260501 13:53:00.448812 1665 raft_consensus.cc:515] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:00.449388 1665 leader_election.cc:290] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5fa23b4772a944c0b0279eabd7476196 (127.0.105.62:44705), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
I20260501 13:53:00.450610 1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "84a39a5f167243ff9bfe07189aa881ca" candidate_term: 5 candidate_status { last_received { term: 4 index: 20 } } ignore_live_leader: false dest_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" is_pre_election: true
W20260501 13:53:00.450801 790 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.62:44705: connect: Connection refused (error 111) [suppressed 5 similar messages]
I20260501 13:53:00.450922 1542 raft_consensus.cc:2468] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84a39a5f167243ff9bfe07189aa881ca in term 4.
I20260501 13:53:00.451649 790 leader_election.cc:304] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7; no voters:
I20260501 13:53:00.451957 1665 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260501 13:53:00.452085 1665 raft_consensus.cc:493] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5fa23b4772a944c0b0279eabd7476196)
I20260501 13:53:00.452194 1665 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 4 FOLLOWER]: Advancing to term 5
W20260501 13:53:00.452518 790 leader_election.cc:336] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5fa23b4772a944c0b0279eabd7476196 (127.0.105.62:44705): Network error: Client connection negotiation failed: client connection to 127.0.105.62:44705: connect: Connection refused (error 111)
I20260501 13:53:00.506127 1665 raft_consensus.cc:515] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:00.506986 1665 leader_election.cc:290] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [CANDIDATE]: Term 5 election: Requested vote from peers 5fa23b4772a944c0b0279eabd7476196 (127.0.105.62:44705), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
I20260501 13:53:00.507876 1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "84a39a5f167243ff9bfe07189aa881ca" candidate_term: 5 candidate_status { last_received { term: 4 index: 20 } } ignore_live_leader: false dest_uuid: "e47102ddbd4f41c1bf0a2864d97237b7"
I20260501 13:53:00.508121 1542 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 4 FOLLOWER]: Advancing to term 5
W20260501 13:53:00.508822 790 leader_election.cc:336] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 5fa23b4772a944c0b0279eabd7476196 (127.0.105.62:44705): Network error: Client connection negotiation failed: client connection to 127.0.105.62:44705: connect: Connection refused (error 111)
I20260501 13:53:00.556407 1070 raft_consensus.cc:1275] T 9fc54df2e5e14019af12a8c0977f967f P e4a8a400d10343718a43d5fcc669dd80 [term 2 FOLLOWER]: Refusing update from remote peer b0226350eae044398f04ef4e5cbb550e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260501 13:53:00.557034 1628 consensus_queue.cc:1048] T 9fc54df2e5e14019af12a8c0977f967f P b0226350eae044398f04ef4e5cbb550e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260501 13:53:00.593770 1542 raft_consensus.cc:2468] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84a39a5f167243ff9bfe07189aa881ca in term 5.
I20260501 13:53:00.596987 790 leader_election.cc:304] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7; no voters: 5fa23b4772a944c0b0279eabd7476196
I20260501 13:53:00.597707 1665 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 5 FOLLOWER]: Leader election won for term 5
I20260501 13:53:00.600319 1209 raft_consensus.cc:1275] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92 [term 2 FOLLOWER]: Refusing update from remote peer b0226350eae044398f04ef4e5cbb550e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260501 13:53:00.600482 1209 pending_rounds.cc:85] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92: Aborting all ops after (but not including) 0
I20260501 13:53:00.600639 1209 pending_rounds.cc:107] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260501 13:53:00.600761 1209 raft_consensus.cc:2889] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260501 13:53:00.601809 1665 raft_consensus.cc:697] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [term 5 LEADER]: Becoming Leader. State: Replica: 84a39a5f167243ff9bfe07189aa881ca, State: Running, Role: LEADER
I20260501 13:53:00.602483 1603 consensus_queue.cc:1048] T 9fc54df2e5e14019af12a8c0977f967f P b0226350eae044398f04ef4e5cbb550e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260501 13:53:00.603387 1665 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 4.20, Last appended by leader: 20, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:00.606772 1673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84a39a5f167243ff9bfe07189aa881ca. Latest consensus state: current_term: 5 leader_uuid: "84a39a5f167243ff9bfe07189aa881ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:53:00.607118 1673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [sys.catalog]: This master's current role is: LEADER
I20260501 13:53:00.608801 1678 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260501 13:53:00.612087 1678 catalog_manager.cc:679] Loaded metadata for table default.table_293c891afe6c4d609f8069858909d21b [id=6d282d8a3f814452977f46b6054268e2]
I20260501 13:53:00.612967 1678 catalog_manager.cc:679] Loaded metadata for table default.table_5a0a455cca3c4ab6a059aac962b7cea0 [id=8818494b487941a4877edf5b079d9756]
I20260501 13:53:00.613440 1678 catalog_manager.cc:679] Loaded metadata for table default.table_d7ed2811199d451c80270ccf0c6c80f4 [id=ac3b885b81d744bfaa69439967b2785f]
I20260501 13:53:00.613832 1678 catalog_manager.cc:679] Loaded metadata for table default.table_5a6d83f7292440209933a379074aadbb [id=b0561cbad3a94637a54b01f351d94fbb]
I20260501 13:53:00.614200 1678 catalog_manager.cc:679] Loaded metadata for table default.table_06eba6fa1b4949ffa139c461e70349f9 [id=ded9bef73cb942de81c7b635d42480f8]
I20260501 13:53:00.614563 1678 catalog_manager.cc:679] Loaded metadata for table default.table_8beb55fb2bdd42309d0183432704b989 [id=f7226f45b9d94af2ad176c2d4a419e88]
I20260501 13:53:00.616194 1678 tablet_loader.cc:96] loaded metadata for tablet 3cb76051f80b49439f8b9191af3cef97 (table default.table_8beb55fb2bdd42309d0183432704b989 [id=f7226f45b9d94af2ad176c2d4a419e88])
I20260501 13:53:00.616741 1678 tablet_loader.cc:96] loaded metadata for tablet 5ac1be5c8f924184956792292483b347 (table default.table_d7ed2811199d451c80270ccf0c6c80f4 [id=ac3b885b81d744bfaa69439967b2785f])
I20260501 13:53:00.617044 1678 tablet_loader.cc:96] loaded metadata for tablet 5c413244650e4784981922b6502f6b97 (table default.table_293c891afe6c4d609f8069858909d21b [id=6d282d8a3f814452977f46b6054268e2])
I20260501 13:53:00.617326 1678 tablet_loader.cc:96] loaded metadata for tablet 8309096d285b4abfbdb7bdbc372be208 (table default.table_5a6d83f7292440209933a379074aadbb [id=b0561cbad3a94637a54b01f351d94fbb])
I20260501 13:53:00.617630 1678 tablet_loader.cc:96] loaded metadata for tablet 9fc54df2e5e14019af12a8c0977f967f (table default.table_06eba6fa1b4949ffa139c461e70349f9 [id=ded9bef73cb942de81c7b635d42480f8])
I20260501 13:53:00.618129 1678 tablet_loader.cc:96] loaded metadata for tablet daa28e4cd0d042b5bb7f7ea04ddb19b7 (table default.table_5a0a455cca3c4ab6a059aac962b7cea0 [id=8818494b487941a4877edf5b079d9756])
I20260501 13:53:00.618386 1678 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260501 13:53:00.619365 1678 catalog_manager.cc:1269] Loaded cluster ID: 6424d308ce1d424b8fe7d2c1b812f4f7
I20260501 13:53:00.619458 1678 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260501 13:53:00.620632 1678 catalog_manager.cc:1514] Loading token signing keys...
I20260501 13:53:00.621486 1678 catalog_manager.cc:6055] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca: Loaded TSK: 0
I20260501 13:53:00.622375 1678 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260501 13:53:00.623205 1678 catalog_manager.cc:1537] Loading latest processed Hive Metastore notification log event ID...
I20260501 13:53:00.624037 1678 catalog_manager.cc:5893] Last processed Hive Metastore notification event ID: -1
I20260501 13:53:00.647539 801 master_service.cc:954] ReplaceTablet: received request to replace tablet 3cb76051f80b49439f8b9191af3cef97 from {username='slave'} at 127.0.0.1:54550
I20260501 13:53:00.647809 800 master_service.cc:954] ReplaceTablet: received request to replace tablet 5c413244650e4784981922b6502f6b97 from {username='slave'} at 127.0.0.1:54550
I20260501 13:53:00.648134 798 master_service.cc:954] ReplaceTablet: received request to replace tablet 9fc54df2e5e14019af12a8c0977f967f from {username='slave'} at 127.0.0.1:54550
I20260501 13:53:00.647271 803 master_service.cc:954] ReplaceTablet: received request to replace tablet 5c413244650e4784981922b6502f6b97 from {username='slave'} at 127.0.0.1:54550
I20260501 13:53:00.649507 799 master_service.cc:954] ReplaceTablet: received request to replace tablet c6507fffd6a4466ab15b1eb89c7f32c7 from {username='slave'} at 127.0.0.1:54550
I20260501 13:53:00.647274 806 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:54550:
table { table_name: "default.Table_06eba6fa1b4949ffa139c461e70349f9" } new_table_name: "default.Table_11735a7200744110a5c2f8e6e42b7b98" modify_external_catalogs: true
I20260501 13:53:00.647588 802 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54550:
name: "default.Table_215a35e1b7404ba1aa928f075959f1e5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260501 13:53:00.649495 805 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54550:
name: "default.Table_7b25f95aeebb4ef29ab513d7047fe028"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260501 13:53:00.649788 797 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:54550:
table { table_name: "default.Table_d7ed2811199d451c80270ccf0c6c80f4" } new_table_name: "default.Table_13d44a18bfde4904b6fd06038e6f31de" modify_external_catalogs: true
I20260501 13:53:00.650542 799 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54550:
name: "default.Table_2e04f40a058b4f08a86ed4ca4cd566cd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260501 13:53:00.649981 804 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54550:
name: "default.Table_b3375377b39846b986b09a1f8fd0b606"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260501 13:53:00.657733 1542 raft_consensus.cc:1275] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 5 FOLLOWER]: Refusing update from remote peer 84a39a5f167243ff9bfe07189aa881ca: Log matching property violated. Preceding OpId in replica: term: 4 index: 20. Preceding OpId from leader: term: 5 index: 22. (index mismatch)
I20260501 13:53:00.658811 1665 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
F20260501 13:53:00.661016 984 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 10 received events: NotificationEvent(eventId=1, eventTime=1777643577, eventType=CREATE_TABLE, dbName=default, tableName=table_5a0a455cca3c4ab6a059aac962b7cea0, message={"server":"","servicePrincipal":"","db":"default","table":"table_5a0a455cca3c4ab6a059aac962b7cea0","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_5a0a455cca3c4ab6a059aac962b7cea0\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1777643577},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/warehouse/table_5a0a455cca3c4ab6a059aac962b7cea0\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_5a0a455cca3c4ab6a059aac962b7cea0\",\"kudu.table_id\":\"8818494b487941a4877edf5b079d9756\",\"transient_lastDdlTime\":\"1777643577\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"6424d308ce1d424b8fe7d2c1b812f4f7\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1777643577,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1777643577, eventType=CREATE_TABLE, dbName=default, tableName=table_86cdd461d6044be8b0ae522dfcc0714a, message={"server":"","servicePrincipal":"","db":"default","table":"table_86cdd461d6044be8b0ae522dfcc0714a","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_86cdd461d6044be8b0ae522dfcc0714a\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1777643577},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/warehouse/table_86cdd461d6044be8b0ae522dfcc0714a\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_86cdd461d6044be8b0ae522dfcc0714a\",\"kudu.table_id\":\"df2e28d52fd341ffac31641fe0482fe8\",\"transient_lastDdlTime\":\"1777643577\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"6424d308ce1d424b8fe7d2c1b812f4f7\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1777643577,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
W20260501 13:53:00.661443 790 consensus_peers.cc:597] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca -> Peer 5fa23b4772a944c0b0279eabd7476196 (127.0.105.62:44705): Couldn't send request to peer 5fa23b4772a944c0b0279eabd7476196. Status: Network error: Client connection negotiation failed: client connection to 127.0.105.62:44705: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
@ 0x7fe2092dfdcd google::LogMessage::Fail() at ??:0
@ 0x7fe2092e3b93 google::LogMessage::SendToLog() at ??:0
@ 0x7fe2092df7cc google::LogMessage::Flush() at ??:0
@ 0x7fe2092dfa11 google::LogMessage::~LogMessage() at ??:0
@ 0x7fe21fd98574 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260501 13:53:00.678232 1632 sys_catalog.cc:455] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84a39a5f167243ff9bfe07189aa881ca. Latest consensus state: current_term: 5 leader_uuid: "84a39a5f167243ff9bfe07189aa881ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:53:00.678488 1632 sys_catalog.cc:458] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [sys.catalog]: This master's current role is: FOLLOWER
@ 0x7fe21fd96e21 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260501 13:53:00.686211 800 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca: Sending DeleteTablet for 3 replicas of tablet 5c413244650e4784981922b6502f6b97
@ 0x7fe21fd9ea9c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260501 13:53:00.688696 800 catalog_manager.cc:6741] ReplaceTablet: tablet 5c413244650e4784981922b6502f6b97 deleted and replaced by tablet 2a199c325704472da2877e30a7f77ac4
I20260501 13:53:00.707767 801 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca: Sending DeleteTablet for 3 replicas of tablet 3cb76051f80b49439f8b9191af3cef97
I20260501 13:53:00.708379 801 catalog_manager.cc:6741] ReplaceTablet: tablet 3cb76051f80b49439f8b9191af3cef97 deleted and replaced by tablet 50e34e638a874a8092a0a4bbeb1418a4
@ 0x7fe21fd9e919 std::_Function_handler<>::_M_invoke() at ??:0
I20260501 13:53:00.717015 801 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54550:
name: "default.Table_94e1f800f024421e91263edf7388fd9f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260501 13:53:00.717180 800 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54550:
name: "default.Table_3e7f39a3248e4963bda4b7790c673d73"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260501 13:53:00.717581 1673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 5 leader_uuid: "84a39a5f167243ff9bfe07189aa881ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:53:00.717800 1673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca [sys.catalog]: This master's current role is: LEADER
I20260501 13:53:00.717754 803 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54550:
table { table_name: "default.Table_5a6d83f7292440209933a379074aadbb" } modify_external_catalogs: true
I20260501 13:53:00.717994 803 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54550:
table { table_name: "default.Table_5a6d83f7292440209933a379074aadbb" } modify_external_catalogs: true
I20260501 13:53:00.719208 798 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 84a39a5f167243ff9bfe07189aa881ca: Sending DeleteTablet for 3 replicas of tablet 9fc54df2e5e14019af12a8c0977f967f
I20260501 13:53:00.719774 798 catalog_manager.cc:6741] ReplaceTablet: tablet 9fc54df2e5e14019af12a8c0977f967f deleted and replaced by tablet cf800b1230eb4b5eac7feb0276d54486
I20260501 13:53:00.720136 798 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54550:
name: "default.Table_49594ae63df046b38ccad889763a2723"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260501 13:53:00.721371 1632 sys_catalog.cc:455] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "84a39a5f167243ff9bfe07189aa881ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:53:00.721594 1632 sys_catalog.cc:458] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [sys.catalog]: This master's current role is: FOLLOWER
I20260501 13:53:00.723380 1323 tablet_service.cc:1558] Processing DeleteTablet for tablet 5c413244650e4784981922b6502f6b97 with delete_type TABLET_DATA_DELETED (replaced by tablet 2a199c325704472da2877e30a7f77ac4) from {username='slave'} at 127.0.0.1:49144
I20260501 13:53:00.723459 1321 tablet_service.cc:1558] Processing DeleteTablet for tablet 9fc54df2e5e14019af12a8c0977f967f with delete_type TABLET_DATA_DELETED (replaced by tablet cf800b1230eb4b5eac7feb0276d54486) from {username='slave'} at 127.0.0.1:49144
I20260501 13:53:00.723697 1322 tablet_service.cc:1558] Processing DeleteTablet for tablet 3cb76051f80b49439f8b9191af3cef97 with delete_type TABLET_DATA_DELETED (replaced by tablet 50e34e638a874a8092a0a4bbeb1418a4) from {username='slave'} at 127.0.0.1:49144
I20260501 13:53:00.737115 1050 tablet_service.cc:1558] Processing DeleteTablet for tablet 5c413244650e4784981922b6502f6b97 with delete_type TABLET_DATA_DELETED (replaced by tablet 2a199c325704472da2877e30a7f77ac4) from {username='slave'} at 127.0.0.1:57976
I20260501 13:53:00.737505 1049 tablet_service.cc:1558] Processing DeleteTablet for tablet 9fc54df2e5e14019af12a8c0977f967f with delete_type TABLET_DATA_DELETED (replaced by tablet cf800b1230eb4b5eac7feb0276d54486) from {username='slave'} at 127.0.0.1:57976
I20260501 13:53:00.737638 1051 tablet_service.cc:1558] Processing DeleteTablet for tablet 3cb76051f80b49439f8b9191af3cef97 with delete_type TABLET_DATA_DELETED (replaced by tablet 50e34e638a874a8092a0a4bbeb1418a4) from {username='slave'} at 127.0.0.1:57976
I20260501 13:53:00.744817 1189 tablet_service.cc:1558] Processing DeleteTablet for tablet 5c413244650e4784981922b6502f6b97 with delete_type TABLET_DATA_DELETED (replaced by tablet 2a199c325704472da2877e30a7f77ac4) from {username='slave'} at 127.0.0.1:42646
I20260501 13:53:00.745167 1187 tablet_service.cc:1558] Processing DeleteTablet for tablet 9fc54df2e5e14019af12a8c0977f967f with delete_type TABLET_DATA_DELETED (replaced by tablet cf800b1230eb4b5eac7feb0276d54486) from {username='slave'} at 127.0.0.1:42646
I20260501 13:53:00.745329 1188 tablet_service.cc:1558] Processing DeleteTablet for tablet 3cb76051f80b49439f8b9191af3cef97 with delete_type TABLET_DATA_DELETED (replaced by tablet 50e34e638a874a8092a0a4bbeb1418a4) from {username='slave'} at 127.0.0.1:42646
@ 0xaa4cad std::function<>::operator()() at ??:?
I20260501 13:53:00.750953 1693 tablet_replica.cc:333] T 5c413244650e4784981922b6502f6b97 P b0226350eae044398f04ef4e5cbb550e: stopping tablet replica
I20260501 13:53:00.751168 1693 raft_consensus.cc:2243] T 5c413244650e4784981922b6502f6b97 P b0226350eae044398f04ef4e5cbb550e [term 1 LEADER]: Raft consensus shutting down.
I20260501 13:53:00.751758 1693 raft_consensus.cc:2272] T 5c413244650e4784981922b6502f6b97 P b0226350eae044398f04ef4e5cbb550e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260501 13:53:00.752295 1693 ts_tablet_manager.cc:1916] T 5c413244650e4784981922b6502f6b97 P b0226350eae044398f04ef4e5cbb550e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260501 13:53:00.755028 1697 tablet_replica.cc:333] T 5c413244650e4784981922b6502f6b97 P e4a8a400d10343718a43d5fcc669dd80: stopping tablet replica
I20260501 13:53:00.755244 1697 raft_consensus.cc:2243] T 5c413244650e4784981922b6502f6b97 P e4a8a400d10343718a43d5fcc669dd80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260501 13:53:00.755420 1699 tablet_replica.cc:333] T 5c413244650e4784981922b6502f6b97 P 1151db12ecae4dd7bb88b3d1415b7d92: stopping tablet replica
I20260501 13:53:00.755517 1697 raft_consensus.cc:2272] T 5c413244650e4784981922b6502f6b97 P e4a8a400d10343718a43d5fcc669dd80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260501 13:53:00.755609 1699 raft_consensus.cc:2243] T 5c413244650e4784981922b6502f6b97 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260501 13:53:00.755837 1699 raft_consensus.cc:2272] T 5c413244650e4784981922b6502f6b97 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260501 13:53:00.756142 1697 ts_tablet_manager.cc:1916] T 5c413244650e4784981922b6502f6b97 P e4a8a400d10343718a43d5fcc669dd80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260501 13:53:00.756388 1699 ts_tablet_manager.cc:1916] T 5c413244650e4784981922b6502f6b97 P 1151db12ecae4dd7bb88b3d1415b7d92: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260501 13:53:00.758733 1321 tablet_service.cc:1511] Processing CreateTablet for tablet 8309096d285b4abfbdb7bdbc372be208 (DEFAULT_TABLE table=default.table_5a6d83f7292440209933a379074aadbb [id=b0561cbad3a94637a54b01f351d94fbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260501 13:53:00.759596 1322 tablet_service.cc:1511] Processing CreateTablet for tablet 5ac1be5c8f924184956792292483b347 (DEFAULT_TABLE table=default.table_d7ed2811199d451c80270ccf0c6c80f4 [id=ac3b885b81d744bfaa69439967b2785f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260501 13:53:00.759878 1320 tablet_service.cc:1511] Processing CreateTablet for tablet 2a199c325704472da2877e30a7f77ac4 (DEFAULT_TABLE table=default.table_293c891afe6c4d609f8069858909d21b [id=6d282d8a3f814452977f46b6054268e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260501 13:53:00.759928 1323 tablet_service.cc:1511] Processing CreateTablet for tablet 50e34e638a874a8092a0a4bbeb1418a4 (DEFAULT_TABLE table=default.table_8beb55fb2bdd42309d0183432704b989 [id=f7226f45b9d94af2ad176c2d4a419e88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260501 13:53:00.760643 1187 tablet_service.cc:1511] Processing CreateTablet for tablet 8309096d285b4abfbdb7bdbc372be208 (DEFAULT_TABLE table=default.table_5a6d83f7292440209933a379074aadbb [id=b0561cbad3a94637a54b01f351d94fbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260501 13:53:00.760793 1188 tablet_service.cc:1511] Processing CreateTablet for tablet 5ac1be5c8f924184956792292483b347 (DEFAULT_TABLE table=default.table_d7ed2811199d451c80270ccf0c6c80f4 [id=ac3b885b81d744bfaa69439967b2785f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260501 13:53:00.761133 1187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8309096d285b4abfbdb7bdbc372be208. 1 dirs total, 0 dirs full, 0 dirs failed
I20260501 13:53:00.761197 1048 tablet_service.cc:1511] Processing CreateTablet for tablet 2a199c325704472da2877e30a7f77ac4 (DEFAULT_TABLE table=default.table_293c891afe6c4d609f8069858909d21b [id=6d282d8a3f814452977f46b6054268e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260501 13:53:00.761686 1048 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a199c325704472da2877e30a7f77ac4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260501 13:53:00.761646 1051 tablet_service.cc:1511] Processing CreateTablet for tablet 5ac1be5c8f924184956792292483b347 (DEFAULT_TABLE table=default.table_d7ed2811199d451c80270ccf0c6c80f4 [id=ac3b885b81d744bfaa69439967b2785f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260501 13:53:00.762110 1051 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ac1be5c8f924184956792292483b347. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7fe20c078529 kudu::Thread::SuperviseThread() at ??:0
I20260501 13:53:00.760792 1186 tablet_service.cc:1511] Processing CreateTablet for tablet 2a199c325704472da2877e30a7f77ac4 (DEFAULT_TABLE table=default.table_293c891afe6c4d609f8069858909d21b [id=6d282d8a3f814452977f46b6054268e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260501 13:53:00.763710 1188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ac1be5c8f924184956792292483b347. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7fe20f6466db start_thread at ??:0
I20260501 13:53:00.767437 1186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a199c325704472da2877e30a7f77ac4. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7fe20819061f clone at ??:0
I20260501 13:53:00.767612 1049 tablet_service.cc:1511] Processing CreateTablet for tablet 8309096d285b4abfbdb7bdbc372be208 (DEFAULT_TABLE table=default.table_5a6d83f7292440209933a379074aadbb [id=b0561cbad3a94637a54b01f351d94fbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260501 13:53:00.767650 1189 tablet_service.cc:1511] Processing CreateTablet for tablet 50e34e638a874a8092a0a4bbeb1418a4 (DEFAULT_TABLE table=default.table_8beb55fb2bdd42309d0183432704b989 [id=f7226f45b9d94af2ad176c2d4a419e88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260501 13:53:00.768064 1189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50e34e638a874a8092a0a4bbeb1418a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260501 13:53:00.768105 1049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8309096d285b4abfbdb7bdbc372be208. 1 dirs total, 0 dirs full, 0 dirs failed
I20260501 13:53:00.779345 1050 tablet_service.cc:1511] Processing CreateTablet for tablet 50e34e638a874a8092a0a4bbeb1418a4 (DEFAULT_TABLE table=default.table_8beb55fb2bdd42309d0183432704b989 [id=f7226f45b9d94af2ad176c2d4a419e88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260501 13:53:00.780406 1702 tablet_bootstrap.cc:492] T 8309096d285b4abfbdb7bdbc372be208 P 1151db12ecae4dd7bb88b3d1415b7d92: Bootstrap starting.
I20260501 13:53:00.781062 1050 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50e34e638a874a8092a0a4bbeb1418a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260501 13:53:00.782305 1702 tablet_bootstrap.cc:654] T 8309096d285b4abfbdb7bdbc372be208 P 1151db12ecae4dd7bb88b3d1415b7d92: Neither blocks nor log segments found. Creating new log.
I20260501 13:53:00.793504 1704 tablet_bootstrap.cc:492] T 2a199c325704472da2877e30a7f77ac4 P e4a8a400d10343718a43d5fcc669dd80: Bootstrap starting.
I20260501 13:53:00.797268 1321 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8309096d285b4abfbdb7bdbc372be208. 1 dirs total, 0 dirs full, 0 dirs failed
I20260501 13:53:00.797864 1322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ac1be5c8f924184956792292483b347. 1 dirs total, 0 dirs full, 0 dirs failed
I20260501 13:53:00.798302 1320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a199c325704472da2877e30a7f77ac4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260501 13:53:00.798722 1323 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50e34e638a874a8092a0a4bbeb1418a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260501 13:53:00.805075 1704 tablet_bootstrap.cc:654] T 2a199c325704472da2877e30a7f77ac4 P e4a8a400d10343718a43d5fcc669dd80: Neither blocks nor log segments found. Creating new log.
I20260501 13:53:00.806242 1702 tablet_bootstrap.cc:492] T 8309096d285b4abfbdb7bdbc372be208 P 1151db12ecae4dd7bb88b3d1415b7d92: No bootstrap required, opened a new log
I20260501 13:53:00.806428 1702 ts_tablet_manager.cc:1403] T 8309096d285b4abfbdb7bdbc372be208 P 1151db12ecae4dd7bb88b3d1415b7d92: Time spent bootstrapping tablet: real 0.026s user 0.004s sys 0.012s
I20260501 13:53:00.807561 1702 raft_consensus.cc:359] T 8309096d285b4abfbdb7bdbc372be208 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:53:00.807884 1702 raft_consensus.cc:385] T 8309096d285b4abfbdb7bdbc372be208 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260501 13:53:00.807989 1702 raft_consensus.cc:740] T 8309096d285b4abfbdb7bdbc372be208 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1151db12ecae4dd7bb88b3d1415b7d92, State: Initialized, Role: FOLLOWER
I20260501 13:53:00.808312 1702 consensus_queue.cc:260] T 8309096d285b4abfbdb7bdbc372be208 P 1151db12ecae4dd7bb88b3d1415b7d92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:53:00.809121 1702 ts_tablet_manager.cc:1434] T 8309096d285b4abfbdb7bdbc372be208 P 1151db12ecae4dd7bb88b3d1415b7d92: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260501 13:53:00.809577 1702 tablet_bootstrap.cc:492] T 2a199c325704472da2877e30a7f77ac4 P 1151db12ecae4dd7bb88b3d1415b7d92: Bootstrap starting.
I20260501 13:53:00.811280 1702 tablet_bootstrap.cc:654] T 2a199c325704472da2877e30a7f77ac4 P 1151db12ecae4dd7bb88b3d1415b7d92: Neither blocks nor log segments found. Creating new log.
I20260501 13:53:00.813208 1702 tablet_bootstrap.cc:492] T 2a199c325704472da2877e30a7f77ac4 P 1151db12ecae4dd7bb88b3d1415b7d92: No bootstrap required, opened a new log
I20260501 13:53:00.813345 1702 ts_tablet_manager.cc:1403] T 2a199c325704472da2877e30a7f77ac4 P 1151db12ecae4dd7bb88b3d1415b7d92: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260501 13:53:00.814188 1702 raft_consensus.cc:359] T 2a199c325704472da2877e30a7f77ac4 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:53:00.814441 1702 raft_consensus.cc:385] T 2a199c325704472da2877e30a7f77ac4 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260501 13:53:00.814568 1702 raft_consensus.cc:740] T 2a199c325704472da2877e30a7f77ac4 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1151db12ecae4dd7bb88b3d1415b7d92, State: Initialized, Role: FOLLOWER
I20260501 13:53:00.814867 1702 consensus_queue.cc:260] T 2a199c325704472da2877e30a7f77ac4 P 1151db12ecae4dd7bb88b3d1415b7d92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:53:00.815575 1702 ts_tablet_manager.cc:1434] T 2a199c325704472da2877e30a7f77ac4 P 1151db12ecae4dd7bb88b3d1415b7d92: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260501 13:53:00.815879 1702 tablet_bootstrap.cc:492] T 5ac1be5c8f924184956792292483b347 P 1151db12ecae4dd7bb88b3d1415b7d92: Bootstrap starting.
I20260501 13:53:00.817584 1702 tablet_bootstrap.cc:654] T 5ac1be5c8f924184956792292483b347 P 1151db12ecae4dd7bb88b3d1415b7d92: Neither blocks nor log segments found. Creating new log.
I20260501 13:53:00.832490 1702 tablet_bootstrap.cc:492] T 5ac1be5c8f924184956792292483b347 P 1151db12ecae4dd7bb88b3d1415b7d92: No bootstrap required, opened a new log
I20260501 13:53:00.832680 1702 ts_tablet_manager.cc:1403] T 5ac1be5c8f924184956792292483b347 P 1151db12ecae4dd7bb88b3d1415b7d92: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.008s
I20260501 13:53:00.833564 1702 raft_consensus.cc:359] T 5ac1be5c8f924184956792292483b347 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:53:00.833801 1702 raft_consensus.cc:385] T 5ac1be5c8f924184956792292483b347 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260501 13:53:00.833884 1702 raft_consensus.cc:740] T 5ac1be5c8f924184956792292483b347 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1151db12ecae4dd7bb88b3d1415b7d92, State: Initialized, Role: FOLLOWER
I20260501 13:53:00.833997 1704 tablet_bootstrap.cc:492] T 2a199c325704472da2877e30a7f77ac4 P e4a8a400d10343718a43d5fcc669dd80: No bootstrap required, opened a new log
I20260501 13:53:00.834148 1704 ts_tablet_manager.cc:1403] T 2a199c325704472da2877e30a7f77ac4 P e4a8a400d10343718a43d5fcc669dd80: Time spent bootstrapping tablet: real 0.041s user 0.007s sys 0.004s
I20260501 13:53:00.834125 1702 consensus_queue.cc:260] T 5ac1be5c8f924184956792292483b347 P 1151db12ecae4dd7bb88b3d1415b7d92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:53:00.834681 1702 ts_tablet_manager.cc:1434] T 5ac1be5c8f924184956792292483b347 P 1151db12ecae4dd7bb88b3d1415b7d92: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260501 13:53:00.835074 1702 tablet_bootstrap.cc:492] T 50e34e638a874a8092a0a4bbeb1418a4 P 1151db12ecae4dd7bb88b3d1415b7d92: Bootstrap starting.
I20260501 13:53:00.835695 1704 raft_consensus.cc:359] T 2a199c325704472da2877e30a7f77ac4 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:53:00.835898 1704 raft_consensus.cc:385] T 2a199c325704472da2877e30a7f77ac4 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260501 13:53:00.835985 1704 raft_consensus.cc:740] T 2a199c325704472da2877e30a7f77ac4 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4a8a400d10343718a43d5fcc669dd80, State: Initialized, Role: FOLLOWER
I20260501 13:53:00.836230 1704 consensus_queue.cc:260] T 2a199c325704472da2877e30a7f77ac4 P e4a8a400d10343718a43d5fcc669dd80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:53:00.836896 1702 tablet_bootstrap.cc:654] T 50e34e638a874a8092a0a4bbeb1418a4 P 1151db12ecae4dd7bb88b3d1415b7d92: Neither blocks nor log segments found. Creating new log.
I20260501 13:53:00.836717 1704 ts_tablet_manager.cc:1434] T 2a199c325704472da2877e30a7f77ac4 P e4a8a400d10343718a43d5fcc669dd80: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260501 13:53:00.838351 1704 tablet_bootstrap.cc:492] T 50e34e638a874a8092a0a4bbeb1418a4 P e4a8a400d10343718a43d5fcc669dd80: Bootstrap starting.
I20260501 13:53:00.840901 1704 tablet_bootstrap.cc:654] T 50e34e638a874a8092a0a4bbeb1418a4 P e4a8a400d10343718a43d5fcc669dd80: Neither blocks nor log segments found. Creating new log.
I20260501 13:53:00.847306 1704 tablet_bootstrap.cc:492] T 50e34e638a874a8092a0a4bbeb1418a4 P e4a8a400d10343718a43d5fcc669dd80: No bootstrap required, opened a new log
I20260501 13:53:00.847502 1704 ts_tablet_manager.cc:1403] T 50e34e638a874a8092a0a4bbeb1418a4 P e4a8a400d10343718a43d5fcc669dd80: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.007s
I20260501 13:53:00.847560 1702 tablet_bootstrap.cc:492] T 50e34e638a874a8092a0a4bbeb1418a4 P 1151db12ecae4dd7bb88b3d1415b7d92: No bootstrap required, opened a new log
I20260501 13:53:00.847751 1702 ts_tablet_manager.cc:1403] T 50e34e638a874a8092a0a4bbeb1418a4 P 1151db12ecae4dd7bb88b3d1415b7d92: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.000s
I20260501 13:53:00.848349 1704 raft_consensus.cc:359] T 50e34e638a874a8092a0a4bbeb1418a4 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:53:00.848515 1702 raft_consensus.cc:359] T 50e34e638a874a8092a0a4bbeb1418a4 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:53:00.848601 1704 raft_consensus.cc:385] T 50e34e638a874a8092a0a4bbeb1418a4 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260501 13:53:00.848711 1704 raft_consensus.cc:740] T 50e34e638a874a8092a0a4bbeb1418a4 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4a8a400d10343718a43d5fcc669dd80, State: Initialized, Role: FOLLOWER
I20260501 13:53:00.848764 1702 raft_consensus.cc:385] T 50e34e638a874a8092a0a4bbeb1418a4 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260501 13:53:00.848881 1702 raft_consensus.cc:740] T 50e34e638a874a8092a0a4bbeb1418a4 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1151db12ecae4dd7bb88b3d1415b7d92, State: Initialized, Role: FOLLOWER
I20260501 13:53:00.849052 1704 consensus_queue.cc:260] T 50e34e638a874a8092a0a4bbeb1418a4 P e4a8a400d10343718a43d5fcc669dd80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:53:00.849187 1702 consensus_queue.cc:260] T 50e34e638a874a8092a0a4bbeb1418a4 P 1151db12ecae4dd7bb88b3d1415b7d92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:53:00.850274 1702 ts_tablet_manager.cc:1434] T 50e34e638a874a8092a0a4bbeb1418a4 P 1151db12ecae4dd7bb88b3d1415b7d92: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260501 13:53:00.851716 1704 ts_tablet_manager.cc:1434] T 50e34e638a874a8092a0a4bbeb1418a4 P e4a8a400d10343718a43d5fcc669dd80: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260501 13:53:00.852068 1704 tablet_bootstrap.cc:492] T 8309096d285b4abfbdb7bdbc372be208 P e4a8a400d10343718a43d5fcc669dd80: Bootstrap starting.
I20260501 13:53:00.853874 1704 tablet_bootstrap.cc:654] T 8309096d285b4abfbdb7bdbc372be208 P e4a8a400d10343718a43d5fcc669dd80: Neither blocks nor log segments found. Creating new log.
I20260501 13:53:00.856851 1704 tablet_bootstrap.cc:492] T 8309096d285b4abfbdb7bdbc372be208 P e4a8a400d10343718a43d5fcc669dd80: No bootstrap required, opened a new log
I20260501 13:53:00.857049 1704 ts_tablet_manager.cc:1403] T 8309096d285b4abfbdb7bdbc372be208 P e4a8a400d10343718a43d5fcc669dd80: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20260501 13:53:00.857908 1704 raft_consensus.cc:359] T 8309096d285b4abfbdb7bdbc372be208 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:53:00.858132 1704 raft_consensus.cc:385] T 8309096d285b4abfbdb7bdbc372be208 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260501 13:53:00.858264 1704 raft_consensus.cc:740] T 8309096d285b4abfbdb7bdbc372be208 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4a8a400d10343718a43d5fcc669dd80, State: Initialized, Role: FOLLOWER
I20260501 13:53:00.858685 1704 consensus_queue.cc:260] T 8309096d285b4abfbdb7bdbc372be208 P e4a8a400d10343718a43d5fcc669dd80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:53:00.859251 1704 ts_tablet_manager.cc:1434] T 8309096d285b4abfbdb7bdbc372be208 P e4a8a400d10343718a43d5fcc669dd80: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260501 13:53:00.893988 1693 ts_tablet_manager.cc:1929] T 5c413244650e4784981922b6502f6b97 P b0226350eae044398f04ef4e5cbb550e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260501 13:53:00.894157 1693 log.cc:1199] T 5c413244650e4784981922b6502f6b97 P b0226350eae044398f04ef4e5cbb550e: Deleting WAL directory at /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-2/wal/wals/5c413244650e4784981922b6502f6b97
I20260501 13:53:00.894644 1693 ts_tablet_manager.cc:1950] T 5c413244650e4784981922b6502f6b97 P b0226350eae044398f04ef4e5cbb550e: Deleting consensus metadata
I20260501 13:53:00.898128 1699 ts_tablet_manager.cc:1929] T 5c413244650e4784981922b6502f6b97 P 1151db12ecae4dd7bb88b3d1415b7d92: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260501 13:53:00.898259 1699 log.cc:1199] T 5c413244650e4784981922b6502f6b97 P 1151db12ecae4dd7bb88b3d1415b7d92: Deleting WAL directory at /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-1/wal/wals/5c413244650e4784981922b6502f6b97
I20260501 13:53:00.898655 1699 ts_tablet_manager.cc:1950] T 5c413244650e4784981922b6502f6b97 P 1151db12ecae4dd7bb88b3d1415b7d92: Deleting consensus metadata
I20260501 13:53:00.900179 1693 tablet_replica.cc:333] T 9fc54df2e5e14019af12a8c0977f967f P b0226350eae044398f04ef4e5cbb550e: stopping tablet replica
I20260501 13:53:00.900394 1693 raft_consensus.cc:2243] T 9fc54df2e5e14019af12a8c0977f967f P b0226350eae044398f04ef4e5cbb550e [term 2 LEADER]: Raft consensus shutting down.
I20260501 13:53:00.900739 1693 raft_consensus.cc:2272] T 9fc54df2e5e14019af12a8c0977f967f P b0226350eae044398f04ef4e5cbb550e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260501 13:53:00.901863 1693 ts_tablet_manager.cc:1916] T 9fc54df2e5e14019af12a8c0977f967f P b0226350eae044398f04ef4e5cbb550e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260501 13:53:00.922470 1714 tablet_bootstrap.cc:492] T 50e34e638a874a8092a0a4bbeb1418a4 P b0226350eae044398f04ef4e5cbb550e: Bootstrap starting.
I20260501 13:53:00.925874 1714 tablet_bootstrap.cc:654] T 50e34e638a874a8092a0a4bbeb1418a4 P b0226350eae044398f04ef4e5cbb550e: Neither blocks nor log segments found. Creating new log.
I20260501 13:53:00.934230 1714 tablet_bootstrap.cc:492] T 50e34e638a874a8092a0a4bbeb1418a4 P b0226350eae044398f04ef4e5cbb550e: No bootstrap required, opened a new log
I20260501 13:53:00.934522 1714 ts_tablet_manager.cc:1403] T 50e34e638a874a8092a0a4bbeb1418a4 P b0226350eae044398f04ef4e5cbb550e: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.007s
I20260501 13:53:00.935658 1714 raft_consensus.cc:359] T 50e34e638a874a8092a0a4bbeb1418a4 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:53:00.935999 1714 raft_consensus.cc:385] T 50e34e638a874a8092a0a4bbeb1418a4 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260501 13:53:00.936149 1714 raft_consensus.cc:740] T 50e34e638a874a8092a0a4bbeb1418a4 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0226350eae044398f04ef4e5cbb550e, State: Initialized, Role: FOLLOWER
I20260501 13:53:00.936527 1714 consensus_queue.cc:260] T 50e34e638a874a8092a0a4bbeb1418a4 P b0226350eae044398f04ef4e5cbb550e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:53:00.941015 1714 ts_tablet_manager.cc:1434] T 50e34e638a874a8092a0a4bbeb1418a4 P b0226350eae044398f04ef4e5cbb550e: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260501 13:53:00.944187 1714 tablet_bootstrap.cc:492] T 5ac1be5c8f924184956792292483b347 P b0226350eae044398f04ef4e5cbb550e: Bootstrap starting.
I20260501 13:53:00.946156 1714 tablet_bootstrap.cc:654] T 5ac1be5c8f924184956792292483b347 P b0226350eae044398f04ef4e5cbb550e: Neither blocks nor log segments found. Creating new log.
I20260501 13:53:00.991562 1699 tablet_replica.cc:333] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92: stopping tablet replica
I20260501 13:53:00.991539 1697 ts_tablet_manager.cc:1929] T 5c413244650e4784981922b6502f6b97 P e4a8a400d10343718a43d5fcc669dd80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260501 13:53:00.991693 1697 log.cc:1199] T 5c413244650e4784981922b6502f6b97 P e4a8a400d10343718a43d5fcc669dd80: Deleting WAL directory at /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-0/wal/wals/5c413244650e4784981922b6502f6b97
I20260501 13:53:00.991793 1699 raft_consensus.cc:2243] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260501 13:53:00.991971 1699 raft_consensus.cc:2272] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260501 13:53:00.993325 1697 ts_tablet_manager.cc:1950] T 5c413244650e4784981922b6502f6b97 P e4a8a400d10343718a43d5fcc669dd80: Deleting consensus metadata
I20260501 13:53:00.998229 1697 tablet_replica.cc:333] T 9fc54df2e5e14019af12a8c0977f967f P e4a8a400d10343718a43d5fcc669dd80: stopping tablet replica
I20260501 13:53:00.998442 1697 raft_consensus.cc:2243] T 9fc54df2e5e14019af12a8c0977f967f P e4a8a400d10343718a43d5fcc669dd80 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260501 13:53:00.998628 1697 raft_consensus.cc:2272] T 9fc54df2e5e14019af12a8c0977f967f P e4a8a400d10343718a43d5fcc669dd80 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260501 13:53:00.999586 1697 ts_tablet_manager.cc:1916] T 9fc54df2e5e14019af12a8c0977f967f P e4a8a400d10343718a43d5fcc669dd80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260501 13:53:00.999634 1699 ts_tablet_manager.cc:1916] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260501 13:53:01.002717 1704 tablet_bootstrap.cc:492] T 5ac1be5c8f924184956792292483b347 P e4a8a400d10343718a43d5fcc669dd80: Bootstrap starting.
I20260501 13:53:01.004534 1704 tablet_bootstrap.cc:654] T 5ac1be5c8f924184956792292483b347 P e4a8a400d10343718a43d5fcc669dd80: Neither blocks nor log segments found. Creating new log.
I20260501 13:53:01.011660 1714 tablet_bootstrap.cc:492] T 5ac1be5c8f924184956792292483b347 P b0226350eae044398f04ef4e5cbb550e: No bootstrap required, opened a new log
I20260501 13:53:01.011863 1714 ts_tablet_manager.cc:1403] T 5ac1be5c8f924184956792292483b347 P b0226350eae044398f04ef4e5cbb550e: Time spent bootstrapping tablet: real 0.068s user 0.017s sys 0.023s
I20260501 13:53:01.012689 1714 raft_consensus.cc:359] T 5ac1be5c8f924184956792292483b347 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:53:01.012909 1714 raft_consensus.cc:385] T 5ac1be5c8f924184956792292483b347 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260501 13:53:01.013012 1714 raft_consensus.cc:740] T 5ac1be5c8f924184956792292483b347 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0226350eae044398f04ef4e5cbb550e, State: Initialized, Role: FOLLOWER
I20260501 13:53:01.020033 1714 consensus_queue.cc:260] T 5ac1be5c8f924184956792292483b347 P b0226350eae044398f04ef4e5cbb550e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:53:01.025107 1656 raft_consensus.cc:493] T 2a199c325704472da2877e30a7f77ac4 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:01.025309 1656 raft_consensus.cc:515] T 2a199c325704472da2877e30a7f77ac4 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:53:01.026134 1656 leader_election.cc:290] T 2a199c325704472da2877e30a7f77ac4 P 1151db12ecae4dd7bb88b3d1415b7d92 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e4a8a400d10343718a43d5fcc669dd80 (127.0.105.1:44349), b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841)
I20260501 13:53:01.026897 1342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a199c325704472da2877e30a7f77ac4" candidate_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0226350eae044398f04ef4e5cbb550e" is_pre_election: true
W20260501 13:53:01.027561 1143 leader_election.cc:343] T 2a199c325704472da2877e30a7f77ac4 P 1151db12ecae4dd7bb88b3d1415b7d92 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Illegal state: must be running to vote when last-logged opid is not known
I20260501 13:53:01.028371 1070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a199c325704472da2877e30a7f77ac4" candidate_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4a8a400d10343718a43d5fcc669dd80" is_pre_election: true
I20260501 13:53:01.028518 1714 ts_tablet_manager.cc:1434] T 5ac1be5c8f924184956792292483b347 P b0226350eae044398f04ef4e5cbb550e: Time spent starting tablet: real 0.017s user 0.002s sys 0.000s
I20260501 13:53:01.028623 1070 raft_consensus.cc:2468] T 2a199c325704472da2877e30a7f77ac4 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1151db12ecae4dd7bb88b3d1415b7d92 in term 0.
I20260501 13:53:01.029109 1144 leader_election.cc:304] T 2a199c325704472da2877e30a7f77ac4 P 1151db12ecae4dd7bb88b3d1415b7d92 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1151db12ecae4dd7bb88b3d1415b7d92, e4a8a400d10343718a43d5fcc669dd80; no voters: b0226350eae044398f04ef4e5cbb550e
I20260501 13:53:01.029584 1656 raft_consensus.cc:2804] T 2a199c325704472da2877e30a7f77ac4 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260501 13:53:01.029719 1656 raft_consensus.cc:493] T 2a199c325704472da2877e30a7f77ac4 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260501 13:53:01.029832 1656 raft_consensus.cc:3060] T 2a199c325704472da2877e30a7f77ac4 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Advancing to term 1
I20260501 13:53:01.032461 1714 tablet_bootstrap.cc:492] T 8309096d285b4abfbdb7bdbc372be208 P b0226350eae044398f04ef4e5cbb550e: Bootstrap starting.
I20260501 13:53:01.034473 1714 tablet_bootstrap.cc:654] T 8309096d285b4abfbdb7bdbc372be208 P b0226350eae044398f04ef4e5cbb550e: Neither blocks nor log segments found. Creating new log.
I20260501 13:53:01.037886 1704 tablet_bootstrap.cc:492] T 5ac1be5c8f924184956792292483b347 P e4a8a400d10343718a43d5fcc669dd80: No bootstrap required, opened a new log
I20260501 13:53:01.038100 1704 ts_tablet_manager.cc:1403] T 5ac1be5c8f924184956792292483b347 P e4a8a400d10343718a43d5fcc669dd80: Time spent bootstrapping tablet: real 0.035s user 0.002s sys 0.006s
I20260501 13:53:01.039023 1704 raft_consensus.cc:359] T 5ac1be5c8f924184956792292483b347 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:53:01.039256 1704 raft_consensus.cc:385] T 5ac1be5c8f924184956792292483b347 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260501 13:53:01.039335 1704 raft_consensus.cc:740] T 5ac1be5c8f924184956792292483b347 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4a8a400d10343718a43d5fcc669dd80, State: Initialized, Role: FOLLOWER
I20260501 13:53:01.039682 1704 consensus_queue.cc:260] T 5ac1be5c8f924184956792292483b347 P e4a8a400d10343718a43d5fcc669dd80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:53:01.041188 1704 ts_tablet_manager.cc:1434] T 5ac1be5c8f924184956792292483b347 P e4a8a400d10343718a43d5fcc669dd80: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260501 13:53:01.045693 1714 tablet_bootstrap.cc:492] T 8309096d285b4abfbdb7bdbc372be208 P b0226350eae044398f04ef4e5cbb550e: No bootstrap required, opened a new log
I20260501 13:53:01.045874 1714 ts_tablet_manager.cc:1403] T 8309096d285b4abfbdb7bdbc372be208 P b0226350eae044398f04ef4e5cbb550e: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.000s
I20260501 13:53:01.046674 1714 raft_consensus.cc:359] T 8309096d285b4abfbdb7bdbc372be208 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:53:01.046856 1714 raft_consensus.cc:385] T 8309096d285b4abfbdb7bdbc372be208 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260501 13:53:01.046936 1714 raft_consensus.cc:740] T 8309096d285b4abfbdb7bdbc372be208 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0226350eae044398f04ef4e5cbb550e, State: Initialized, Role: FOLLOWER
I20260501 13:53:01.047170 1714 consensus_queue.cc:260] T 8309096d285b4abfbdb7bdbc372be208 P b0226350eae044398f04ef4e5cbb550e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:53:01.050710 1714 ts_tablet_manager.cc:1434] T 8309096d285b4abfbdb7bdbc372be208 P b0226350eae044398f04ef4e5cbb550e: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260501 13:53:01.053980 1714 tablet_bootstrap.cc:492] T 2a199c325704472da2877e30a7f77ac4 P b0226350eae044398f04ef4e5cbb550e: Bootstrap starting.
I20260501 13:53:01.055832 1714 tablet_bootstrap.cc:654] T 2a199c325704472da2877e30a7f77ac4 P b0226350eae044398f04ef4e5cbb550e: Neither blocks nor log segments found. Creating new log.
I20260501 13:53:01.078075 1714 tablet_bootstrap.cc:492] T 2a199c325704472da2877e30a7f77ac4 P b0226350eae044398f04ef4e5cbb550e: No bootstrap required, opened a new log
I20260501 13:53:01.078301 1714 ts_tablet_manager.cc:1403] T 2a199c325704472da2877e30a7f77ac4 P b0226350eae044398f04ef4e5cbb550e: Time spent bootstrapping tablet: real 0.024s user 0.008s sys 0.014s
I20260501 13:53:01.079252 1714 raft_consensus.cc:359] T 2a199c325704472da2877e30a7f77ac4 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:53:01.079488 1714 raft_consensus.cc:385] T 2a199c325704472da2877e30a7f77ac4 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260501 13:53:01.079596 1714 raft_consensus.cc:740] T 2a199c325704472da2877e30a7f77ac4 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0226350eae044398f04ef4e5cbb550e, State: Initialized, Role: FOLLOWER
I20260501 13:53:01.079936 1714 consensus_queue.cc:260] T 2a199c325704472da2877e30a7f77ac4 P b0226350eae044398f04ef4e5cbb550e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:53:01.081499 1714 ts_tablet_manager.cc:1434] T 2a199c325704472da2877e30a7f77ac4 P b0226350eae044398f04ef4e5cbb550e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260501 13:53:01.093791 1604 raft_consensus.cc:493] T 5ac1be5c8f924184956792292483b347 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:01.094008 1604 raft_consensus.cc:515] T 5ac1be5c8f924184956792292483b347 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:53:01.094749 1604 leader_election.cc:290] T 5ac1be5c8f924184956792292483b347 P b0226350eae044398f04ef4e5cbb550e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e4a8a400d10343718a43d5fcc669dd80 (127.0.105.1:44349), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
I20260501 13:53:01.095680 1070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac1be5c8f924184956792292483b347" candidate_uuid: "b0226350eae044398f04ef4e5cbb550e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4a8a400d10343718a43d5fcc669dd80" is_pre_election: true
I20260501 13:53:01.095947 1070 raft_consensus.cc:2468] T 5ac1be5c8f924184956792292483b347 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0226350eae044398f04ef4e5cbb550e in term 0.
I20260501 13:53:01.096359 1209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac1be5c8f924184956792292483b347" candidate_uuid: "b0226350eae044398f04ef4e5cbb550e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" is_pre_election: true
I20260501 13:53:01.096603 1209 raft_consensus.cc:2468] T 5ac1be5c8f924184956792292483b347 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0226350eae044398f04ef4e5cbb550e in term 0.
I20260501 13:53:01.097242 1278 leader_election.cc:304] T 5ac1be5c8f924184956792292483b347 P b0226350eae044398f04ef4e5cbb550e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0226350eae044398f04ef4e5cbb550e, e4a8a400d10343718a43d5fcc669dd80; no voters:
I20260501 13:53:01.097798 1604 raft_consensus.cc:2804] T 5ac1be5c8f924184956792292483b347 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260501 13:53:01.097911 1604 raft_consensus.cc:493] T 5ac1be5c8f924184956792292483b347 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260501 13:53:01.097992 1604 raft_consensus.cc:3060] T 5ac1be5c8f924184956792292483b347 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Advancing to term 1
I20260501 13:53:01.100032 1693 ts_tablet_manager.cc:1929] T 9fc54df2e5e14019af12a8c0977f967f P b0226350eae044398f04ef4e5cbb550e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.1
I20260501 13:53:01.100167 1693 log.cc:1199] T 9fc54df2e5e14019af12a8c0977f967f P b0226350eae044398f04ef4e5cbb550e: Deleting WAL directory at /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-2/wal/wals/9fc54df2e5e14019af12a8c0977f967f
I20260501 13:53:01.100548 1693 ts_tablet_manager.cc:1950] T 9fc54df2e5e14019af12a8c0977f967f P b0226350eae044398f04ef4e5cbb550e: Deleting consensus metadata
I20260501 13:53:01.103034 1697 ts_tablet_manager.cc:1929] T 9fc54df2e5e14019af12a8c0977f967f P e4a8a400d10343718a43d5fcc669dd80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.1
I20260501 13:53:01.103164 1697 log.cc:1199] T 9fc54df2e5e14019af12a8c0977f967f P e4a8a400d10343718a43d5fcc669dd80: Deleting WAL directory at /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-0/wal/wals/9fc54df2e5e14019af12a8c0977f967f
I20260501 13:53:01.103497 1693 tablet_replica.cc:333] T 3cb76051f80b49439f8b9191af3cef97 P b0226350eae044398f04ef4e5cbb550e: stopping tablet replica
I20260501 13:53:01.103581 1697 ts_tablet_manager.cc:1950] T 9fc54df2e5e14019af12a8c0977f967f P e4a8a400d10343718a43d5fcc669dd80: Deleting consensus metadata
I20260501 13:53:01.103469 1656 raft_consensus.cc:515] T 2a199c325704472da2877e30a7f77ac4 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:53:01.103677 1693 raft_consensus.cc:2243] T 3cb76051f80b49439f8b9191af3cef97 P b0226350eae044398f04ef4e5cbb550e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260501 13:53:01.103864 1693 raft_consensus.cc:2272] T 3cb76051f80b49439f8b9191af3cef97 P b0226350eae044398f04ef4e5cbb550e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260501 13:53:01.104178 1656 leader_election.cc:290] T 2a199c325704472da2877e30a7f77ac4 P 1151db12ecae4dd7bb88b3d1415b7d92 [CANDIDATE]: Term 1 election: Requested vote from peers e4a8a400d10343718a43d5fcc669dd80 (127.0.105.1:44349), b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841)
I20260501 13:53:01.104424 1693 ts_tablet_manager.cc:1916] T 3cb76051f80b49439f8b9191af3cef97 P b0226350eae044398f04ef4e5cbb550e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260501 13:53:01.104692 1628 raft_consensus.cc:493] T 8309096d285b4abfbdb7bdbc372be208 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:01.104861 1628 raft_consensus.cc:515] T 8309096d285b4abfbdb7bdbc372be208 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:53:01.105474 1342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a199c325704472da2877e30a7f77ac4" candidate_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0226350eae044398f04ef4e5cbb550e"
I20260501 13:53:01.105656 1342 raft_consensus.cc:3060] T 2a199c325704472da2877e30a7f77ac4 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Advancing to term 1
I20260501 13:53:01.105693 1628 leader_election.cc:290] T 8309096d285b4abfbdb7bdbc372be208 P b0226350eae044398f04ef4e5cbb550e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e4a8a400d10343718a43d5fcc669dd80 (127.0.105.1:44349), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
I20260501 13:53:01.108193 1209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8309096d285b4abfbdb7bdbc372be208" candidate_uuid: "b0226350eae044398f04ef4e5cbb550e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" is_pre_election: true
I20260501 13:53:01.108426 1209 raft_consensus.cc:2468] T 8309096d285b4abfbdb7bdbc372be208 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0226350eae044398f04ef4e5cbb550e in term 0.
I20260501 13:53:01.108548 1071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a199c325704472da2877e30a7f77ac4" candidate_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4a8a400d10343718a43d5fcc669dd80"
I20260501 13:53:01.108750 1071 raft_consensus.cc:3060] T 2a199c325704472da2877e30a7f77ac4 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Advancing to term 1
I20260501 13:53:01.108901 1070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8309096d285b4abfbdb7bdbc372be208" candidate_uuid: "b0226350eae044398f04ef4e5cbb550e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4a8a400d10343718a43d5fcc669dd80" is_pre_election: true
I20260501 13:53:01.108953 1279 leader_election.cc:304] T 8309096d285b4abfbdb7bdbc372be208 P b0226350eae044398f04ef4e5cbb550e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e; no voters:
I20260501 13:53:01.109056 1070 raft_consensus.cc:2468] T 8309096d285b4abfbdb7bdbc372be208 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0226350eae044398f04ef4e5cbb550e in term 0.
I20260501 13:53:01.109233 1628 raft_consensus.cc:2804] T 8309096d285b4abfbdb7bdbc372be208 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260501 13:53:01.109350 1628 raft_consensus.cc:493] T 8309096d285b4abfbdb7bdbc372be208 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260501 13:53:01.109460 1628 raft_consensus.cc:3060] T 8309096d285b4abfbdb7bdbc372be208 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Advancing to term 1
I20260501 13:53:01.111595 1699 ts_tablet_manager.cc:1929] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.1
I20260501 13:53:01.111799 1699 log.cc:1199] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92: Deleting WAL directory at /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-1/wal/wals/9fc54df2e5e14019af12a8c0977f967f
I20260501 13:53:01.112277 1699 ts_tablet_manager.cc:1950] T 9fc54df2e5e14019af12a8c0977f967f P 1151db12ecae4dd7bb88b3d1415b7d92: Deleting consensus metadata
W20260501 13:53:01.119333 1277 connection.cc:570] server connection from 127.0.0.1:49144 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260501 13:53:01.119333 1005 connection.cc:570] server connection from 127.0.0.1:57976 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260501 13:53:01.119791 1145 connection.cc:570] server connection from 127.0.0.1:42646 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260501 13:53:01.120108 1279 connection.cc:570] client connection to 127.0.105.61:33209 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260501 13:53:01.120174 1401 connection.cc:570] client connection to 127.0.105.61:33209 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260501 13:53:01.120407 1145 connection.cc:570] client connection to 127.0.105.61:33209 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260501 13:53:01.120939 1401 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.105.61:33209)
W20260501 13:53:01.122704 1401 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.0.105.61:33209)
W20260501 13:53:01.122915 1007 connection.cc:570] client connection to 127.0.105.61:33209 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260501 13:53:01.123728 1676 negotiation.cc:336] Failed RPC negotiation. Trace:
0501 13:53:01.121987 (+ 0us) reactor.cc:678] Submitting negotiation task for client connection to 127.0.105.61:33209 (local address 127.0.105.2:45401)
0501 13:53:01.122247 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0501 13:53:01.122266 (+ 19us) client_negotiation.cc:175] Beginning negotiation
0501 13:53:01.122402 (+ 136us) client_negotiation.cc:253] Sending NEGOTIATE NegotiatePB request
0501 13:53:01.122643 (+ 241us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":133}
I20260501 13:53:01.197633 1697 tablet_replica.cc:333] T 3cb76051f80b49439f8b9191af3cef97 P e4a8a400d10343718a43d5fcc669dd80: stopping tablet replica
W20260501 13:53:01.197589 1005 connection.cc:463] server connection from 127.0.0.1:57976 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57976 (request call id 2) could send its response
I20260501 13:53:01.197939 1697 raft_consensus.cc:2243] T 3cb76051f80b49439f8b9191af3cef97 P e4a8a400d10343718a43d5fcc669dd80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260501 13:53:01.198163 1697 raft_consensus.cc:2272] T 3cb76051f80b49439f8b9191af3cef97 P e4a8a400d10343718a43d5fcc669dd80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260501 13:53:01.198772 1697 ts_tablet_manager.cc:1916] T 3cb76051f80b49439f8b9191af3cef97 P e4a8a400d10343718a43d5fcc669dd80: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260501 13:53:01.199604 1145 connection.cc:463] server connection from 127.0.0.1:42646 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42646 (request call id 2) could send its response
I20260501 13:53:01.200217 1699 tablet_replica.cc:333] T 3cb76051f80b49439f8b9191af3cef97 P 1151db12ecae4dd7bb88b3d1415b7d92: stopping tablet replica
I20260501 13:53:01.200412 1699 raft_consensus.cc:2243] T 3cb76051f80b49439f8b9191af3cef97 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 1 LEADER]: Raft consensus shutting down.
I20260501 13:53:01.200788 1699 raft_consensus.cc:2272] T 3cb76051f80b49439f8b9191af3cef97 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260501 13:53:01.222110 1342 raft_consensus.cc:2468] T 2a199c325704472da2877e30a7f77ac4 P b0226350eae044398f04ef4e5cbb550e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1151db12ecae4dd7bb88b3d1415b7d92 in term 1.
I20260501 13:53:01.222304 1699 ts_tablet_manager.cc:1916] T 3cb76051f80b49439f8b9191af3cef97 P 1151db12ecae4dd7bb88b3d1415b7d92: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260501 13:53:01.222698 1604 raft_consensus.cc:515] T 5ac1be5c8f924184956792292483b347 P b0226350eae044398f04ef4e5cbb550e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:53:01.222836 1628 raft_consensus.cc:515] T 8309096d285b4abfbdb7bdbc372be208 P b0226350eae044398f04ef4e5cbb550e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:53:01.223515 1604 leader_election.cc:290] T 5ac1be5c8f924184956792292483b347 P b0226350eae044398f04ef4e5cbb550e [CANDIDATE]: Term 1 election: Requested vote from peers e4a8a400d10343718a43d5fcc669dd80 (127.0.105.1:44349), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
I20260501 13:53:01.223547 1628 leader_election.cc:290] T 8309096d285b4abfbdb7bdbc372be208 P b0226350eae044398f04ef4e5cbb550e [CANDIDATE]: Term 1 election: Requested vote from peers e4a8a400d10343718a43d5fcc669dd80 (127.0.105.1:44349), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
I20260501 13:53:01.224274 1143 leader_election.cc:304] T 2a199c325704472da2877e30a7f77ac4 P 1151db12ecae4dd7bb88b3d1415b7d92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e; no voters:
I20260501 13:53:01.224644 1656 raft_consensus.cc:2804] T 2a199c325704472da2877e30a7f77ac4 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 1 FOLLOWER]: Leader election won for term 1
I20260501 13:53:01.225260 1209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac1be5c8f924184956792292483b347" candidate_uuid: "b0226350eae044398f04ef4e5cbb550e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1151db12ecae4dd7bb88b3d1415b7d92"
I20260501 13:53:01.225574 1209 raft_consensus.cc:3060] T 5ac1be5c8f924184956792292483b347 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Advancing to term 1
I20260501 13:53:01.226241 1668 raft_consensus.cc:493] T 50e34e638a874a8092a0a4bbeb1418a4 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:01.226401 1668 raft_consensus.cc:515] T 50e34e638a874a8092a0a4bbeb1418a4 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:53:01.227772 1207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50e34e638a874a8092a0a4bbeb1418a4" candidate_uuid: "e4a8a400d10343718a43d5fcc669dd80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" is_pre_election: true
I20260501 13:53:01.228107 1207 raft_consensus.cc:2468] T 50e34e638a874a8092a0a4bbeb1418a4 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4a8a400d10343718a43d5fcc669dd80 in term 0.
I20260501 13:53:01.228217 1656 raft_consensus.cc:697] T 2a199c325704472da2877e30a7f77ac4 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 1 LEADER]: Becoming Leader. State: Replica: 1151db12ecae4dd7bb88b3d1415b7d92, State: Running, Role: LEADER
I20260501 13:53:01.228536 1656 consensus_queue.cc:237] T 2a199c325704472da2877e30a7f77ac4 P 1151db12ecae4dd7bb88b3d1415b7d92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:53:01.227185 1668 leader_election.cc:290] T 50e34e638a874a8092a0a4bbeb1418a4 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565), b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841)
I20260501 13:53:01.228976 1007 leader_election.cc:304] T 50e34e638a874a8092a0a4bbeb1418a4 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1151db12ecae4dd7bb88b3d1415b7d92, e4a8a400d10343718a43d5fcc669dd80; no voters:
I20260501 13:53:01.229008 1208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8309096d285b4abfbdb7bdbc372be208" candidate_uuid: "b0226350eae044398f04ef4e5cbb550e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1151db12ecae4dd7bb88b3d1415b7d92"
I20260501 13:53:01.229212 1668 raft_consensus.cc:2804] T 50e34e638a874a8092a0a4bbeb1418a4 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260501 13:53:01.229275 1208 raft_consensus.cc:3060] T 8309096d285b4abfbdb7bdbc372be208 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Advancing to term 1
I20260501 13:53:01.229300 1668 raft_consensus.cc:493] T 50e34e638a874a8092a0a4bbeb1418a4 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260501 13:53:01.229367 1668 raft_consensus.cc:3060] T 50e34e638a874a8092a0a4bbeb1418a4 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Advancing to term 1
I20260501 13:53:01.231180 1071 raft_consensus.cc:2468] T 2a199c325704472da2877e30a7f77ac4 P e4a8a400d10343718a43d5fcc669dd80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1151db12ecae4dd7bb88b3d1415b7d92 in term 1.
I20260501 13:53:01.232347 1070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8309096d285b4abfbdb7bdbc372be208" candidate_uuid: "b0226350eae044398f04ef4e5cbb550e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4a8a400d10343718a43d5fcc669dd80"
I20260501 13:53:01.233325 1070 raft_consensus.cc:3060] T 8309096d285b4abfbdb7bdbc372be208 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Advancing to term 1
I20260501 13:53:01.234371 1723 raft_consensus.cc:493] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 84a39a5f167243ff9bfe07189aa881ca)
I20260501 13:53:01.234532 1723 raft_consensus.cc:515] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:01.234597 1071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac1be5c8f924184956792292483b347" candidate_uuid: "b0226350eae044398f04ef4e5cbb550e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4a8a400d10343718a43d5fcc669dd80"
I20260501 13:53:01.234772 1071 raft_consensus.cc:3060] T 5ac1be5c8f924184956792292483b347 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Advancing to term 1
I20260501 13:53:01.235904 1342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50e34e638a874a8092a0a4bbeb1418a4" candidate_uuid: "e4a8a400d10343718a43d5fcc669dd80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0226350eae044398f04ef4e5cbb550e" is_pre_election: true
I20260501 13:53:01.236192 1342 raft_consensus.cc:2468] T 50e34e638a874a8092a0a4bbeb1418a4 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4a8a400d10343718a43d5fcc669dd80 in term 0.
I20260501 13:53:01.241703 1723 leader_election.cc:290] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5fa23b4772a944c0b0279eabd7476196 (127.0.105.62:44705), 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209)
W20260501 13:53:01.242908 1517 leader_election.cc:336] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 5fa23b4772a944c0b0279eabd7476196 (127.0.105.62:44705): Network error: Client connection negotiation failed: client connection to 127.0.105.62:44705: connect: Connection refused (error 111)
W20260501 13:53:01.243126 1520 leader_election.cc:336] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
I20260501 13:53:01.243324 1520 leader_election.cc:304] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e47102ddbd4f41c1bf0a2864d97237b7; no voters: 5fa23b4772a944c0b0279eabd7476196, 84a39a5f167243ff9bfe07189aa881ca
I20260501 13:53:01.243722 1723 raft_consensus.cc:2749] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260501 13:53:01.321254 1208 raft_consensus.cc:2468] T 8309096d285b4abfbdb7bdbc372be208 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0226350eae044398f04ef4e5cbb550e in term 1.
I20260501 13:53:01.321324 1668 raft_consensus.cc:515] T 50e34e638a874a8092a0a4bbeb1418a4 P e4a8a400d10343718a43d5fcc669dd80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:53:01.321830 1209 raft_consensus.cc:2468] T 5ac1be5c8f924184956792292483b347 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0226350eae044398f04ef4e5cbb550e in term 1.
I20260501 13:53:01.322178 1668 leader_election.cc:290] T 50e34e638a874a8092a0a4bbeb1418a4 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 election: Requested vote from peers 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565), b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841)
I20260501 13:53:01.324887 1070 raft_consensus.cc:2468] T 8309096d285b4abfbdb7bdbc372be208 P e4a8a400d10343718a43d5fcc669dd80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0226350eae044398f04ef4e5cbb550e in term 1.
I20260501 13:53:01.325253 1693 ts_tablet_manager.cc:1929] T 3cb76051f80b49439f8b9191af3cef97 P b0226350eae044398f04ef4e5cbb550e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260501 13:53:01.325423 1693 log.cc:1199] T 3cb76051f80b49439f8b9191af3cef97 P b0226350eae044398f04ef4e5cbb550e: Deleting WAL directory at /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-2/wal/wals/3cb76051f80b49439f8b9191af3cef97
I20260501 13:53:01.325676 1278 leader_election.cc:304] T 8309096d285b4abfbdb7bdbc372be208 P b0226350eae044398f04ef4e5cbb550e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0226350eae044398f04ef4e5cbb550e, e4a8a400d10343718a43d5fcc669dd80; no voters:
I20260501 13:53:01.325508 1342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50e34e638a874a8092a0a4bbeb1418a4" candidate_uuid: "e4a8a400d10343718a43d5fcc669dd80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0226350eae044398f04ef4e5cbb550e"
I20260501 13:53:01.326082 1628 raft_consensus.cc:2804] T 8309096d285b4abfbdb7bdbc372be208 P b0226350eae044398f04ef4e5cbb550e [term 1 FOLLOWER]: Leader election won for term 1
I20260501 13:53:01.326179 1342 raft_consensus.cc:3060] T 50e34e638a874a8092a0a4bbeb1418a4 P b0226350eae044398f04ef4e5cbb550e [term 0 FOLLOWER]: Advancing to term 1
I20260501 13:53:01.326254 1628 raft_consensus.cc:697] T 8309096d285b4abfbdb7bdbc372be208 P b0226350eae044398f04ef4e5cbb550e [term 1 LEADER]: Becoming Leader. State: Replica: b0226350eae044398f04ef4e5cbb550e, State: Running, Role: LEADER
I20260501 13:53:01.326308 1209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50e34e638a874a8092a0a4bbeb1418a4" candidate_uuid: "e4a8a400d10343718a43d5fcc669dd80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1151db12ecae4dd7bb88b3d1415b7d92"
I20260501 13:53:01.326342 1279 leader_election.cc:304] T 5ac1be5c8f924184956792292483b347 P b0226350eae044398f04ef4e5cbb550e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e; no voters:
I20260501 13:53:01.326475 1209 raft_consensus.cc:3060] T 50e34e638a874a8092a0a4bbeb1418a4 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 0 FOLLOWER]: Advancing to term 1
I20260501 13:53:01.326566 1628 consensus_queue.cc:237] T 8309096d285b4abfbdb7bdbc372be208 P b0226350eae044398f04ef4e5cbb550e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:53:01.326675 1604 raft_consensus.cc:2804] T 5ac1be5c8f924184956792292483b347 P b0226350eae044398f04ef4e5cbb550e [term 1 FOLLOWER]: Leader election won for term 1
I20260501 13:53:01.326799 1604 raft_consensus.cc:697] T 5ac1be5c8f924184956792292483b347 P b0226350eae044398f04ef4e5cbb550e [term 1 LEADER]: Becoming Leader. State: Replica: b0226350eae044398f04ef4e5cbb550e, State: Running, Role: LEADER
I20260501 13:53:01.327088 1604 consensus_queue.cc:237] T 5ac1be5c8f924184956792292483b347 P b0226350eae044398f04ef4e5cbb550e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:53:01.437438 1693 ts_tablet_manager.cc:1950] T 3cb76051f80b49439f8b9191af3cef97 P b0226350eae044398f04ef4e5cbb550e: Deleting consensus metadata
W20260501 13:53:01.438831 1729 log.cc:927] Time spent T 5ac1be5c8f924184956792292483b347 P b0226350eae044398f04ef4e5cbb550e: Append to log took a long time: real 0.109s user 0.000s sys 0.000s
I20260501 13:53:01.439774 1342 raft_consensus.cc:2468] T 50e34e638a874a8092a0a4bbeb1418a4 P b0226350eae044398f04ef4e5cbb550e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4a8a400d10343718a43d5fcc669dd80 in term 1.
W20260501 13:53:01.440105 1728 log.cc:927] Time spent T 8309096d285b4abfbdb7bdbc372be208 P b0226350eae044398f04ef4e5cbb550e: Append to log took a long time: real 0.110s user 0.000s sys 0.000s
I20260501 13:53:01.440474 1209 raft_consensus.cc:2468] T 50e34e638a874a8092a0a4bbeb1418a4 P 1151db12ecae4dd7bb88b3d1415b7d92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4a8a400d10343718a43d5fcc669dd80 in term 1.
I20260501 13:53:01.440835 1699 ts_tablet_manager.cc:1929] T 3cb76051f80b49439f8b9191af3cef97 P 1151db12ecae4dd7bb88b3d1415b7d92: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260501 13:53:01.440958 1699 log.cc:1199] T 3cb76051f80b49439f8b9191af3cef97 P 1151db12ecae4dd7bb88b3d1415b7d92: Deleting WAL directory at /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-1/wal/wals/3cb76051f80b49439f8b9191af3cef97
W20260501 13:53:01.440968 1277 connection.cc:463] server connection from 127.0.0.1:49144 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49144 (request call id 1) could send its response
I20260501 13:53:01.441372 1699 ts_tablet_manager.cc:1950] T 3cb76051f80b49439f8b9191af3cef97 P 1151db12ecae4dd7bb88b3d1415b7d92: Deleting consensus metadata
I20260501 13:53:01.441619 1007 leader_election.cc:304] T 50e34e638a874a8092a0a4bbeb1418a4 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1151db12ecae4dd7bb88b3d1415b7d92, e4a8a400d10343718a43d5fcc669dd80; no voters:
I20260501 13:53:01.441679 1071 raft_consensus.cc:2468] T 5ac1be5c8f924184956792292483b347 P e4a8a400d10343718a43d5fcc669dd80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0226350eae044398f04ef4e5cbb550e in term 1.
I20260501 13:53:01.442030 1668 raft_consensus.cc:2804] T 50e34e638a874a8092a0a4bbeb1418a4 P e4a8a400d10343718a43d5fcc669dd80 [term 1 FOLLOWER]: Leader election won for term 1
W20260501 13:53:01.442457 1145 connection.cc:463] server connection from 127.0.0.1:42646 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42646 (request call id 1) could send its response
I20260501 13:53:01.442921 1668 raft_consensus.cc:697] T 50e34e638a874a8092a0a4bbeb1418a4 P e4a8a400d10343718a43d5fcc669dd80 [term 1 LEADER]: Becoming Leader. State: Replica: e4a8a400d10343718a43d5fcc669dd80, State: Running, Role: LEADER
I20260501 13:53:01.443356 1668 consensus_queue.cc:237] T 50e34e638a874a8092a0a4bbeb1418a4 P e4a8a400d10343718a43d5fcc669dd80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } }
I20260501 13:53:01.537813 1697 ts_tablet_manager.cc:1929] T 3cb76051f80b49439f8b9191af3cef97 P e4a8a400d10343718a43d5fcc669dd80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260501 13:53:01.537961 1697 log.cc:1199] T 3cb76051f80b49439f8b9191af3cef97 P e4a8a400d10343718a43d5fcc669dd80: Deleting WAL directory at /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/ts-0/wal/wals/3cb76051f80b49439f8b9191af3cef97
I20260501 13:53:01.538451 1697 ts_tablet_manager.cc:1950] T 3cb76051f80b49439f8b9191af3cef97 P e4a8a400d10343718a43d5fcc669dd80: Deleting consensus metadata
W20260501 13:53:01.540457 1005 connection.cc:463] server connection from 127.0.0.1:57976 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57976 (request call id 1) could send its response
I20260501 13:53:01.767815 1736 raft_consensus.cc:493] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 84a39a5f167243ff9bfe07189aa881ca)
I20260501 13:53:01.768024 1736 raft_consensus.cc:515] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:01.768729 1736 leader_election.cc:290] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5fa23b4772a944c0b0279eabd7476196 (127.0.105.62:44705), 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209)
W20260501 13:53:01.770948 1517 leader_election.cc:336] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 5fa23b4772a944c0b0279eabd7476196 (127.0.105.62:44705): Network error: Client connection negotiation failed: client connection to 127.0.105.62:44705: connect: Connection refused (error 111)
W20260501 13:53:01.771142 1520 leader_election.cc:336] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
I20260501 13:53:01.771323 1520 leader_election.cc:304] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e47102ddbd4f41c1bf0a2864d97237b7; no voters: 5fa23b4772a944c0b0279eabd7476196, 84a39a5f167243ff9bfe07189aa881ca
I20260501 13:53:01.771620 1736 raft_consensus.cc:2749] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260501 13:53:01.837291 420 external_mini_cluster.cc:1366] Running /tmp/dist-test-task4ezAfy/build/asan/bin/kudu
/tmp/dist-test-task4ezAfy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.62:44705
--webserver_interface=127.0.105.62
--webserver_port=45017
--builtin_ntp_servers=127.0.105.20:38455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729
--hive_metastore_uris=thrift://127.0.0.1:33681
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260501 13:53:02.111404 1628 consensus_queue.cc:1048] T 8309096d285b4abfbdb7bdbc372be208 P b0226350eae044398f04ef4e5cbb550e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260501 13:53:02.118227 1747 consensus_queue.cc:1048] T 8309096d285b4abfbdb7bdbc372be208 P b0226350eae044398f04ef4e5cbb550e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260501 13:53:02.200639 1742 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260501 13:53:02.200954 1742 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260501 13:53:02.201049 1742 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260501 13:53:02.203024 1750 consensus_queue.cc:1048] T 5ac1be5c8f924184956792292483b347 P b0226350eae044398f04ef4e5cbb550e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260501 13:53:02.208871 1628 consensus_queue.cc:1048] T 5ac1be5c8f924184956792292483b347 P b0226350eae044398f04ef4e5cbb550e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260501 13:53:02.214517 1742 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260501 13:53:02.214639 1742 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260501 13:53:02.214738 1742 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260501 13:53:02.214785 1742 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260501 13:53:02.225692 1742 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.105.20:38455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:33681
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.62:44705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.105.62
--webserver_port=45017
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision aa7cc0aa9c559b87f264235eaadd03ec492277e4
build type FASTDEBUG
built by None at 01 May 2026 13:43:23 UTC on bdcb31816ec0
build id 11694
ASAN enabled
I20260501 13:53:02.227383 1742 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260501 13:53:02.229143 1742 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20260501 13:53:02.232282 1745 consensus_queue.cc:1048] T 2a199c325704472da2877e30a7f77ac4 P 1151db12ecae4dd7bb88b3d1415b7d92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260501 13:53:02.239018 1761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260501 13:53:02.241714 1762 consensus_queue.cc:1048] T 2a199c325704472da2877e30a7f77ac4 P 1151db12ecae4dd7bb88b3d1415b7d92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260501 13:53:02.244360 1763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260501 13:53:02.248291 1742 server_base.cc:1061] running on GCE node
W20260501 13:53:02.249013 1768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260501 13:53:02.250010 1742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260501 13:53:02.252004 1742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260501 13:53:02.253260 1742 hybrid_clock.cc:648] HybridClock initialized: now 1777643582253207 us; error 44 us; skew 500 ppm
I20260501 13:53:02.253677 1742 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260501 13:53:02.256119 1742 webserver.cc:492] Webserver started at http://127.0.105.62:45017/ using document root <none> and password file <none>
I20260501 13:53:02.256901 1742 fs_manager.cc:362] Metadata directory not provided
I20260501 13:53:02.257047 1742 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260501 13:53:02.262550 1742 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260501 13:53:02.265592 1774 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260501 13:53:02.266973 1742 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260501 13:53:02.267158 1742 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data,/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/wal
uuid: "5fa23b4772a944c0b0279eabd7476196"
format_stamp: "Formatted at 2026-05-01 13:52:53 on dist-test-slave-qlwv"
I20260501 13:53:02.267954 1742 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260501 13:53:02.293960 1742 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260501 13:53:02.294878 1742 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260501 13:53:02.295217 1742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260501 13:53:02.316187 1742 rpc_server.cc:307] RPC server started. Bound to: 127.0.105.62:44705
I20260501 13:53:02.316216 1825 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.105.62:44705 every 8 connection(s)
I20260501 13:53:02.317806 1742 server_base.cc:1193] Dumped server information to /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data/info.pb
I20260501 13:53:02.325239 1826 sys_catalog.cc:263] Verifying existing consensus state
I20260501 13:53:02.328274 420 external_mini_cluster.cc:1428] Started /tmp/dist-test-task4ezAfy/build/asan/bin/kudu as pid 1742
I20260501 13:53:02.329533 1826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: Bootstrap starting.
I20260501 13:53:02.345539 1826 log.cc:826] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: Log is configured to *not* fsync() on all Append() calls
I20260501 13:53:02.367204 1826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: Bootstrap replayed 1/1 log segments. Stats: ops{read=20 overwritten=0 applied=20 ignored=0} inserts{seen=17 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260501 13:53:02.367800 1826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: Bootstrap complete.
I20260501 13:53:02.373220 1826 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 4 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:02.373980 1826 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fa23b4772a944c0b0279eabd7476196, State: Initialized, Role: FOLLOWER
I20260501 13:53:02.374562 1826 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 20, Last appended: 4.20, Last appended by leader: 20, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:02.375391 1831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 4 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:53:02.375674 1831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: This master's current role is: FOLLOWER
I20260501 13:53:02.376451 1826 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: configured and running, proceeding with master startup.
I20260501 13:53:02.403299 1845 catalog_manager.cc:1269] Loaded cluster ID: 6424d308ce1d424b8fe7d2c1b812f4f7
I20260501 13:53:02.403409 1845 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: loading cluster ID for follower catalog manager: success
I20260501 13:53:02.407652 1845 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: acquiring CA information for follower catalog manager: success
I20260501 13:53:02.410179 1845 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260501 13:53:02.410552 1742 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:53:02.416291 1742 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:53:02.465461 1849 raft_consensus.cc:493] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 84a39a5f167243ff9bfe07189aa881ca)
I20260501 13:53:02.465721 1849 raft_consensus.cc:515] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:02.466403 1849 leader_election.cc:290] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5fa23b4772a944c0b0279eabd7476196 (127.0.105.62:44705), 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209)
W20260501 13:53:02.468788 1520 leader_election.cc:336] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
I20260501 13:53:02.474164 1801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" candidate_term: 6 candidate_status { last_received { term: 5 index: 25 } } ignore_live_leader: false dest_uuid: "5fa23b4772a944c0b0279eabd7476196" is_pre_election: true
I20260501 13:53:02.474841 1801 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e47102ddbd4f41c1bf0a2864d97237b7 in term 4.
I20260501 13:53:02.475431 1517 leader_election.cc:304] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196, e47102ddbd4f41c1bf0a2864d97237b7; no voters: 84a39a5f167243ff9bfe07189aa881ca
I20260501 13:53:02.475771 1849 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260501 13:53:02.475910 1849 raft_consensus.cc:493] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 84a39a5f167243ff9bfe07189aa881ca)
I20260501 13:53:02.476020 1849 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 5 FOLLOWER]: Advancing to term 6
I20260501 13:53:02.478128 1849 raft_consensus.cc:515] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:02.478766 1849 leader_election.cc:290] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [CANDIDATE]: Term 6 election: Requested vote from peers 5fa23b4772a944c0b0279eabd7476196 (127.0.105.62:44705), 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209)
I20260501 13:53:02.479242 1801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" candidate_term: 6 candidate_status { last_received { term: 5 index: 25 } } ignore_live_leader: false dest_uuid: "5fa23b4772a944c0b0279eabd7476196"
I20260501 13:53:02.479446 1801 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 4 FOLLOWER]: Advancing to term 6
W20260501 13:53:02.480252 1520 leader_election.cc:336] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
I20260501 13:53:02.485177 1801 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47102ddbd4f41c1bf0a2864d97237b7 in term 6.
I20260501 13:53:02.485674 1517 leader_election.cc:304] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196, e47102ddbd4f41c1bf0a2864d97237b7; no voters: 84a39a5f167243ff9bfe07189aa881ca
I20260501 13:53:02.486035 1849 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 6 FOLLOWER]: Leader election won for term 6
I20260501 13:53:02.486557 1849 raft_consensus.cc:697] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 6 LEADER]: Becoming Leader. State: Replica: e47102ddbd4f41c1bf0a2864d97237b7, State: Running, Role: LEADER
I20260501 13:53:02.487000 1849 consensus_queue.cc:237] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 5.25, Last appended by leader: 25, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:02.488795 1853 sys_catalog.cc:455] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e47102ddbd4f41c1bf0a2864d97237b7. Latest consensus state: current_term: 6 leader_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:53:02.489121 1853 sys_catalog.cc:458] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [sys.catalog]: This master's current role is: LEADER
I20260501 13:53:02.489794 1855 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260501 13:53:02.492270 1855 catalog_manager.cc:679] Loaded metadata for table default.table_293c891afe6c4d609f8069858909d21b [id=6d282d8a3f814452977f46b6054268e2]
I20260501 13:53:02.492805 1855 catalog_manager.cc:679] Loaded metadata for table default.table_5a0a455cca3c4ab6a059aac962b7cea0 [id=8818494b487941a4877edf5b079d9756]
I20260501 13:53:02.493242 1855 catalog_manager.cc:679] Loaded metadata for table default.table_d7ed2811199d451c80270ccf0c6c80f4 [id=ac3b885b81d744bfaa69439967b2785f]
I20260501 13:53:02.493659 1855 catalog_manager.cc:679] Loaded metadata for table default.table_5a6d83f7292440209933a379074aadbb [id=b0561cbad3a94637a54b01f351d94fbb]
I20260501 13:53:02.494025 1855 catalog_manager.cc:679] Loaded metadata for table default.table_06eba6fa1b4949ffa139c461e70349f9 [id=ded9bef73cb942de81c7b635d42480f8]
I20260501 13:53:02.494385 1855 catalog_manager.cc:679] Loaded metadata for table default.table_8beb55fb2bdd42309d0183432704b989 [id=f7226f45b9d94af2ad176c2d4a419e88]
I20260501 13:53:02.496536 1855 tablet_loader.cc:96] loaded metadata for tablet 2a199c325704472da2877e30a7f77ac4 (table default.table_293c891afe6c4d609f8069858909d21b [id=6d282d8a3f814452977f46b6054268e2])
I20260501 13:53:02.497181 1855 tablet_loader.cc:96] loaded metadata for tablet 50e34e638a874a8092a0a4bbeb1418a4 (table default.table_8beb55fb2bdd42309d0183432704b989 [id=f7226f45b9d94af2ad176c2d4a419e88])
I20260501 13:53:02.497867 1855 tablet_loader.cc:96] loaded metadata for tablet 5ac1be5c8f924184956792292483b347 (table default.table_d7ed2811199d451c80270ccf0c6c80f4 [id=ac3b885b81d744bfaa69439967b2785f])
I20260501 13:53:02.498570 1855 tablet_loader.cc:96] loaded metadata for tablet 8309096d285b4abfbdb7bdbc372be208 (table default.table_5a6d83f7292440209933a379074aadbb [id=b0561cbad3a94637a54b01f351d94fbb])
I20260501 13:53:02.499400 1855 tablet_loader.cc:96] loaded metadata for tablet cf800b1230eb4b5eac7feb0276d54486 (table default.table_06eba6fa1b4949ffa139c461e70349f9 [id=ded9bef73cb942de81c7b635d42480f8])
I20260501 13:53:02.499789 1855 tablet_loader.cc:96] loaded metadata for tablet daa28e4cd0d042b5bb7f7ea04ddb19b7 (table default.table_5a0a455cca3c4ab6a059aac962b7cea0 [id=8818494b487941a4877edf5b079d9756])
I20260501 13:53:02.500109 1855 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260501 13:53:02.501641 1855 catalog_manager.cc:1269] Loaded cluster ID: 6424d308ce1d424b8fe7d2c1b812f4f7
I20260501 13:53:02.501719 1855 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260501 13:53:02.503296 1855 catalog_manager.cc:1514] Loading token signing keys...
I20260501 13:53:02.504381 1855 catalog_manager.cc:6055] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7: Loaded TSK: 0
I20260501 13:53:02.505491 1855 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260501 13:53:02.507006 1855 catalog_manager.cc:1537] Loading latest processed Hive Metastore notification log event ID...
I20260501 13:53:02.508100 1855 catalog_manager.cc:5893] Last processed Hive Metastore notification event ID: -1
I20260501 13:53:02.559458 1533 master_service.cc:954] ReplaceTablet: received request to replace tablet 50e34e638a874a8092a0a4bbeb1418a4 from {username='slave'} at 127.0.0.1:35798
I20260501 13:53:02.563385 1532 master_service.cc:954] ReplaceTablet: received request to replace tablet 9fc54df2e5e14019af12a8c0977f967f from {username='slave'} at 127.0.0.1:35798
I20260501 13:53:02.564139 1530 master_service.cc:954] ReplaceTablet: received request to replace tablet 2a199c325704472da2877e30a7f77ac4 from {username='slave'} at 127.0.0.1:35798
I20260501 13:53:02.564945 1529 master_service.cc:954] ReplaceTablet: received request to replace tablet 2a199c325704472da2877e30a7f77ac4 from {username='slave'} at 127.0.0.1:35798
I20260501 13:53:02.568878 1531 master_service.cc:954] ReplaceTablet: received request to replace tablet 8309096d285b4abfbdb7bdbc372be208 from {username='slave'} at 127.0.0.1:35798
I20260501 13:53:02.569340 1751 consensus_queue.cc:1048] T 50e34e638a874a8092a0a4bbeb1418a4 P e4a8a400d10343718a43d5fcc669dd80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260501 13:53:02.584198 1255 heartbeater.cc:344] Connected to a master server at 127.0.105.62:44705
I20260501 13:53:02.589408 1790 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" instance_seqno: 1777643576483530) as {username='slave'} at 127.0.105.2:60025; Asking this server to re-register.
I20260501 13:53:02.566313 1526 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35798:
table { table_name: "default.Table_8beb55fb2bdd42309d0183432704b989" } new_table_name: "default.Table_62e7535549cf414f8af163f82d73b382" modify_external_catalogs: true
I20260501 13:53:02.566551 1525 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35798:
table { table_name: "default.Table_5a0a455cca3c4ab6a059aac962b7cea0" } new_table_name: "default.Table_38e9fe72f57e4e9e927e2fd11e3dc180" modify_external_catalogs: true
I20260501 13:53:02.566854 1524 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35798:
name: "default.Table_2ff058e2a2324419b03d708582f033d5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260501 13:53:02.566077 1527 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35798:
table { table_name: "default.Table_293c891afe6c4d609f8069858909d21b" } new_table_name: "default.Table_06273705ee5b4dff82f6d1ad5c14fac4" modify_external_catalogs: true
I20260501 13:53:02.565586 1528 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35798:
name: "default.Table_058c3f74f9b04b9d92dcd4667e109a5d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260501 13:53:02.571069 1532 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35798:
name: "default.Table_215a35e1b7404ba1aa928f075959f1e5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260501 13:53:02.601536 1255 heartbeater.cc:461] Registering TS with master...
F20260501 13:53:02.602545 1585 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 11 received events: NotificationEvent(eventId=1, eventTime=1777643577, eventType=CREATE_TABLE, dbName=default, tableName=table_5a0a455cca3c4ab6a059aac962b7cea0, message={"server":"","servicePrincipal":"","db":"default","table":"table_5a0a455cca3c4ab6a059aac962b7cea0","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_5a0a455cca3c4ab6a059aac962b7cea0\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1777643577},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/warehouse/table_5a0a455cca3c4ab6a059aac962b7cea0\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_5a0a455cca3c4ab6a059aac962b7cea0\",\"kudu.table_id\":\"8818494b487941a4877edf5b079d9756\",\"transient_lastDdlTime\":\"1777643577\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"6424d308ce1d424b8fe7d2c1b812f4f7\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1777643577,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1777643577, eventType=CREATE_TABLE, dbName=default, tableName=table_86cdd461d6044be8b0ae522dfcc0714a, message={"server":"","servicePrincipal":"","db":"default","table":"table_86cdd461d6044be8b0ae522dfcc0714a","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_86cdd461d6044be8b0ae522dfcc0714a\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1777643577},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/warehouse/table_86cdd461d6044be8b0ae522dfcc0714a\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_86cdd461d6044be8b0ae522dfcc0714a\",\"kudu.table_id\":\"df2e28d52fd341ffac31641fe0482fe8\",\"transient_lastDdlTime\":\"1777643577\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"6424d308ce1d424b8fe7d2c1b812f4f7\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1777643577,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
I20260501 13:53:02.604252 1791 ts_manager.cc:194] Registered new tserver with Master: 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
@ 0x7faf6de80dcd google::LogMessage::Fail() at ??:0
@ 0x7faf6de84b93 google::LogMessage::SendToLog() at ??:0
@ 0x7faf6de807cc google::LogMessage::Flush() at ??:0
@ 0x7faf6de80a11 google::LogMessage::~LogMessage() at ??:0
W20260501 13:53:02.620241 1520 consensus_peers.cc:597] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 -> Peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Couldn't send request to peer 84a39a5f167243ff9bfe07189aa881ca. Status: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260501 13:53:02.620645 1801 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 6 FOLLOWER]: Refusing update from remote peer e47102ddbd4f41c1bf0a2864d97237b7: Log matching property violated. Preceding OpId in replica: term: 4 index: 20. Preceding OpId from leader: term: 6 index: 27. (index mismatch)
I20260501 13:53:02.621557 1853 consensus_queue.cc:1048] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 20, Time since last communication: 0.000s
@ 0x7faf84939574 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7faf84937e21 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260501 13:53:02.640844 1751 consensus_queue.cc:1048] T 50e34e638a874a8092a0a4bbeb1418a4 P e4a8a400d10343718a43d5fcc669dd80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
@ 0x7faf8493fa9c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7faf8493f919 std::_Function_handler<>::_M_invoke() at ??:0
I20260501 13:53:02.664395 1389 heartbeater.cc:344] Connected to a master server at 127.0.105.62:44705
@ 0xaa4cad std::function<>::operator()() at ??:?
I20260501 13:53:02.667380 1117 heartbeater.cc:344] Connected to a master server at 127.0.105.62:44705
@ 0x7faf70c19529 kudu::Thread::SuperviseThread() at ??:0
@ 0x7faf741e76db start_thread at ??:0
@ 0x7faf6cd3161f clone at ??:0
I20260501 13:53:02.725572 1790 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" instance_seqno: 1777643576999180) as {username='slave'} at 127.0.105.3:38667; Asking this server to re-register.
I20260501 13:53:02.725986 1862 mvcc.cc:204] Tried to move back new op lower bound from 7281228106646654976 to 7281228106159513600. Current Snapshot: MvccSnapshot[applied={T|T < 7281228106370744320 or (T in {7281228106646654976})}]
I20260501 13:53:02.726377 1867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 6 leader_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:53:02.726799 1867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: This master's current role is: FOLLOWER
I20260501 13:53:02.726966 1791 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" instance_seqno: 1777643575975284) as {username='slave'} at 127.0.105.1:42905; Asking this server to re-register.
I20260501 13:53:02.726886 1831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e47102ddbd4f41c1bf0a2864d97237b7. Latest consensus state: current_term: 6 leader_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:53:02.727171 1831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: This master's current role is: FOLLOWER
I20260501 13:53:02.728134 1117 heartbeater.cc:461] Registering TS with master...
I20260501 13:53:02.729800 1791 ts_manager.cc:194] Registered new tserver with Master: e4a8a400d10343718a43d5fcc669dd80 (127.0.105.1:44349)
I20260501 13:53:02.731598 1389 heartbeater.cc:461] Registering TS with master...
I20260501 13:53:02.735793 1791 ts_manager.cc:194] Registered new tserver with Master: b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841)
W20260501 13:53:02.950991 1398 connection.cc:570] client connection to 127.0.105.60:39729 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260501 13:53:02.951663 1398 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.105.60:39729)
W20260501 13:53:02.951939 1398 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.105.60:39729)
W20260501 13:53:02.952348 1398 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.0.105.60:39729)
W20260501 13:53:02.952948 1398 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.105.60:39729)
W20260501 13:53:02.953784 1778 connection.cc:570] server connection from 127.0.0.1:41500 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260501 13:53:02.954543 1735 negotiation.cc:336] Failed RPC negotiation. Trace:
0501 13:53:02.951801 (+ 0us) reactor.cc:678] Submitting negotiation task for client connection to 127.0.105.60:39729 (local address 127.0.105.2:54131)
0501 13:53:02.954269 (+ 2468us) negotiation.cc:107] Waiting for socket to connect
0501 13:53:02.954349 (+ 80us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":2340}
I20260501 13:53:03.125136 1831 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e47102ddbd4f41c1bf0a2864d97237b7)
I20260501 13:53:03.125312 1831 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:03.127249 1831 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:53:03.128053 1778 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:53:03.129460 1775 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
W20260501 13:53:03.129694 1778 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
I20260501 13:53:03.129840 1778 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:53:03.130112 1831 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260501 13:53:03.664999 1892 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e47102ddbd4f41c1bf0a2864d97237b7)
I20260501 13:53:03.665174 1892 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:03.665879 1892 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:53:03.667953 1778 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:53:03.668116 1775 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:53:03.668274 1775 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:53:03.668524 1892 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260501 13:53:04.206382 1905 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e47102ddbd4f41c1bf0a2864d97237b7)
I20260501 13:53:04.206595 1905 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:04.207427 1905 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:53:04.209458 1778 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:53:04.209704 1775 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:53:04.209882 1775 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:53:04.210215 1905 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260501 13:53:04.594900 420 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-task4ezAfy/build/asan/bin/kudu pid 1742
I20260501 13:53:04.603432 1821 generic_service.cc:196] Checking for leaks (request via RPC)
I20260501 13:53:04.712356 420 external_mini_cluster.cc:1658] Killing /tmp/dist-test-task4ezAfy/build/asan/bin/kudu with pid 1742
I20260501 13:53:05.687134 420 external_mini_cluster.cc:1366] Running /tmp/dist-test-task4ezAfy/build/asan/bin/kudu
/tmp/dist-test-task4ezAfy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.62:44705
--webserver_interface=127.0.105.62
--webserver_port=45017
--builtin_ntp_servers=127.0.105.20:38455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729
--hive_metastore_uris=thrift://127.0.0.1:33681
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260501 13:53:06.027171 1931 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260501 13:53:06.027619 1931 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260501 13:53:06.027751 1931 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260501 13:53:06.036530 1931 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260501 13:53:06.036718 1931 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260501 13:53:06.036844 1931 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260501 13:53:06.036931 1931 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260501 13:53:06.047797 1931 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.105.20:38455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:33681
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.62:44705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.105.62
--webserver_port=45017
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision aa7cc0aa9c559b87f264235eaadd03ec492277e4
build type FASTDEBUG
built by None at 01 May 2026 13:43:23 UTC on bdcb31816ec0
build id 11694
ASAN enabled
I20260501 13:53:06.049525 1931 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260501 13:53:06.051287 1931 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260501 13:53:06.059811 1941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260501 13:53:06.059871 1938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260501 13:53:06.059850 1939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260501 13:53:06.061321 1931 server_base.cc:1061] running on GCE node
I20260501 13:53:06.062125 1931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260501 13:53:06.063799 1931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260501 13:53:06.065063 1931 hybrid_clock.cc:648] HybridClock initialized: now 1777643586065004 us; error 59 us; skew 500 ppm
I20260501 13:53:06.065438 1931 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260501 13:53:06.067786 1931 webserver.cc:492] Webserver started at http://127.0.105.62:45017/ using document root <none> and password file <none>
I20260501 13:53:06.068552 1931 fs_manager.cc:362] Metadata directory not provided
I20260501 13:53:06.068675 1931 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260501 13:53:06.074144 1931 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260501 13:53:06.077567 1947 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260501 13:53:06.078951 1931 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260501 13:53:06.079145 1931 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data,/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/wal
uuid: "5fa23b4772a944c0b0279eabd7476196"
format_stamp: "Formatted at 2026-05-01 13:52:53 on dist-test-slave-qlwv"
I20260501 13:53:06.079934 1931 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260501 13:53:06.097498 1931 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260501 13:53:06.098446 1931 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260501 13:53:06.098760 1931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260501 13:53:06.119805 1931 rpc_server.cc:307] RPC server started. Bound to: 127.0.105.62:44705
I20260501 13:53:06.119837 2000 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.105.62:44705 every 8 connection(s)
I20260501 13:53:06.121443 1931 server_base.cc:1193] Dumped server information to /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data/info.pb
I20260501 13:53:06.126224 420 external_mini_cluster.cc:1428] Started /tmp/dist-test-task4ezAfy/build/asan/bin/kudu as pid 1931
I20260501 13:53:06.129058 2001 sys_catalog.cc:263] Verifying existing consensus state
I20260501 13:53:06.134102 2001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: Bootstrap starting.
I20260501 13:53:06.145834 2001 log.cc:826] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: Log is configured to *not* fsync() on all Append() calls
I20260501 13:53:06.169061 2001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: Bootstrap replayed 1/1 log segments. Stats: ops{read=28 overwritten=0 applied=25 ignored=0} inserts{seen=20 ignored=0} mutations{seen=16 ignored=0} orphaned_commits=0. Pending: 3 replicates
I20260501 13:53:06.169642 2001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: Bootstrap complete.
I20260501 13:53:06.175117 2001 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 6 FOLLOWER]: Replica starting. Triggering 3 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:06.177977 2001 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 6 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fa23b4772a944c0b0279eabd7476196, State: Initialized, Role: FOLLOWER
I20260501 13:53:06.178686 2001 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 25, Last appended: 6.28, Last appended by leader: 28, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:06.179484 2006 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 6 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:53:06.179764 2006 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: This master's current role is: FOLLOWER
I20260501 13:53:06.180521 2001 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: configured and running, proceeding with master startup.
I20260501 13:53:06.192384 2020 catalog_manager.cc:1269] Loaded cluster ID: 6424d308ce1d424b8fe7d2c1b812f4f7
I20260501 13:53:06.192799 2020 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: loading cluster ID for follower catalog manager: success
I20260501 13:53:06.198300 2020 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: acquiring CA information for follower catalog manager: success
I20260501 13:53:06.204345 2020 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260501 13:53:06.212937 1931 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:53:06.218545 1931 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:53:06.221939 1389 heartbeater.cc:344] Connected to a master server at 127.0.105.62:44705
I20260501 13:53:06.224792 1966 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" instance_seqno: 1777643576999180) as {username='slave'} at 127.0.105.3:35641; Asking this server to re-register.
I20260501 13:53:06.225450 1389 heartbeater.cc:461] Registering TS with master...
I20260501 13:53:06.227144 1966 ts_manager.cc:194] Registered new tserver with Master: b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841)
I20260501 13:53:06.521452 1255 heartbeater.cc:344] Connected to a master server at 127.0.105.62:44705
I20260501 13:53:06.522765 1966 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" instance_seqno: 1777643576483530) as {username='slave'} at 127.0.105.2:51031; Asking this server to re-register.
I20260501 13:53:06.523387 1255 heartbeater.cc:461] Registering TS with master...
I20260501 13:53:06.524756 1966 ts_manager.cc:194] Registered new tserver with Master: 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
I20260501 13:53:06.608292 1117 heartbeater.cc:344] Connected to a master server at 127.0.105.62:44705
I20260501 13:53:06.609844 1966 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" instance_seqno: 1777643575975284) as {username='slave'} at 127.0.105.1:36949; Asking this server to re-register.
I20260501 13:53:06.610402 1117 heartbeater.cc:461] Registering TS with master...
I20260501 13:53:06.611590 1966 ts_manager.cc:194] Registered new tserver with Master: e4a8a400d10343718a43d5fcc669dd80 (127.0.105.1:44349)
I20260501 13:53:06.673110 2006 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:06.673369 2006 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:06.675204 2006 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:53:06.676020 1948 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
W20260501 13:53:06.677237 1951 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:53:06.677449 1948 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:53:06.677604 1948 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:53:06.678006 2006 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260501 13:53:07.381587 2034 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:07.381814 2034 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:07.382480 2034 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:53:07.385103 1951 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:53:07.385357 1948 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:53:07.385507 1948 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:53:07.385744 2034 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260501 13:53:07.888950 2042 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:07.889159 2042 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:07.889870 2042 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:53:07.892120 1951 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:53:07.892329 1948 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:53:07.892482 1948 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:53:07.892731 2042 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260501 13:53:09.086145 2060 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:09.086345 2060 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:09.087046 2060 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:53:09.089141 1951 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:53:09.089329 1948 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:53:09.089462 1948 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:53:09.089677 2060 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260501 13:53:09.178107 420 external_mini_cluster.cc:1366] Running /tmp/dist-test-task4ezAfy/build/asan/bin/kudu
/tmp/dist-test-task4ezAfy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.60:39729
--webserver_interface=127.0.105.60
--webserver_port=43413
--builtin_ntp_servers=127.0.105.20:38455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729
--hive_metastore_uris=thrift://127.0.0.1:33681
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260501 13:53:09.526477 2066 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260501 13:53:09.526919 2066 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260501 13:53:09.527065 2066 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260501 13:53:09.537448 2066 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260501 13:53:09.537588 2066 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260501 13:53:09.537729 2066 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260501 13:53:09.537817 2066 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260501 13:53:09.549491 2066 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.105.20:38455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:33681
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.60:39729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.105.60
--webserver_port=43413
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision aa7cc0aa9c559b87f264235eaadd03ec492277e4
build type FASTDEBUG
built by None at 01 May 2026 13:43:23 UTC on bdcb31816ec0
build id 11694
ASAN enabled
I20260501 13:53:09.551457 2066 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260501 13:53:09.553375 2066 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260501 13:53:09.562302 2077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260501 13:53:09.562338 2076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260501 13:53:09.563640 2079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260501 13:53:09.564496 2066 server_base.cc:1061] running on GCE node
I20260501 13:53:09.565274 2066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260501 13:53:09.566721 2066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260501 13:53:09.567951 2066 hybrid_clock.cc:648] HybridClock initialized: now 1777643589567910 us; error 48 us; skew 500 ppm
I20260501 13:53:09.568400 2066 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260501 13:53:09.570678 2066 webserver.cc:492] Webserver started at http://127.0.105.60:43413/ using document root <none> and password file <none>
I20260501 13:53:09.571583 2066 fs_manager.cc:362] Metadata directory not provided
I20260501 13:53:09.571702 2066 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260501 13:53:09.577200 2066 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260501 13:53:09.580120 2085 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260501 13:53:09.581554 2066 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260501 13:53:09.581744 2066 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/data,/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/wal
uuid: "e47102ddbd4f41c1bf0a2864d97237b7"
format_stamp: "Formatted at 2026-05-01 13:52:54 on dist-test-slave-qlwv"
I20260501 13:53:09.582473 2066 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260501 13:53:09.604403 2066 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260501 13:53:09.605299 2066 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260501 13:53:09.605599 2066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260501 13:53:09.627856 2066 rpc_server.cc:307] RPC server started. Bound to: 127.0.105.60:39729
I20260501 13:53:09.627875 2138 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.105.60:39729 every 8 connection(s)
I20260501 13:53:09.629506 2066 server_base.cc:1193] Dumped server information to /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-2/data/info.pb
I20260501 13:53:09.636772 2139 sys_catalog.cc:263] Verifying existing consensus state
I20260501 13:53:09.638346 420 external_mini_cluster.cc:1428] Started /tmp/dist-test-task4ezAfy/build/asan/bin/kudu as pid 2066
I20260501 13:53:09.641299 2139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7: Bootstrap starting.
I20260501 13:53:09.654721 2139 log.cc:826] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7: Log is configured to *not* fsync() on all Append() calls
I20260501 13:53:09.679647 2139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7: Bootstrap replayed 1/1 log segments. Stats: ops{read=29 overwritten=0 applied=25 ignored=0} inserts{seen=20 ignored=0} mutations{seen=16 ignored=0} orphaned_commits=0. Pending: 4 replicates
I20260501 13:53:09.680456 2139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7: Bootstrap complete.
I20260501 13:53:09.687470 2139 raft_consensus.cc:359] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 6 FOLLOWER]: Replica starting. Triggering 4 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:09.690773 2139 raft_consensus.cc:740] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 6 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47102ddbd4f41c1bf0a2864d97237b7, State: Initialized, Role: FOLLOWER
I20260501 13:53:09.691593 2139 consensus_queue.cc:260] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 25, Last appended: 6.29, Last appended by leader: 29, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:09.692391 2144 sys_catalog.cc:455] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 6 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:53:09.692621 2144 sys_catalog.cc:458] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [sys.catalog]: This master's current role is: FOLLOWER
I20260501 13:53:09.693593 2139 sys_catalog.cc:565] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260501 13:53:09.707293 2158 catalog_manager.cc:1269] Loaded cluster ID: 6424d308ce1d424b8fe7d2c1b812f4f7
I20260501 13:53:09.707523 2158 catalog_manager.cc:1562] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7: loading cluster ID for follower catalog manager: success
I20260501 13:53:09.712800 2158 catalog_manager.cc:1584] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7: acquiring CA information for follower catalog manager: success
I20260501 13:53:09.715693 2158 catalog_manager.cc:1612] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260501 13:53:09.734299 1388 heartbeater.cc:344] Connected to a master server at 127.0.105.60:39729
I20260501 13:53:09.730226 2066 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:53:09.736191 2066 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:53:09.738013 2104 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" instance_seqno: 1777643576999180) as {username='slave'} at 127.0.105.3:47023; Asking this server to re-register.
I20260501 13:53:09.738924 1388 heartbeater.cc:461] Registering TS with master...
I20260501 13:53:09.740870 2104 ts_manager.cc:194] Registered new tserver with Master: b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841)
I20260501 13:53:10.033064 1254 heartbeater.cc:344] Connected to a master server at 127.0.105.60:39729
I20260501 13:53:10.034613 2103 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" instance_seqno: 1777643576483530) as {username='slave'} at 127.0.105.2:42689; Asking this server to re-register.
I20260501 13:53:10.035283 1254 heartbeater.cc:461] Registering TS with master...
I20260501 13:53:10.036506 2103 ts_manager.cc:194] Registered new tserver with Master: 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
I20260501 13:53:10.120512 1116 heartbeater.cc:344] Connected to a master server at 127.0.105.60:39729
I20260501 13:53:10.122184 2103 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" instance_seqno: 1777643575975284) as {username='slave'} at 127.0.105.1:54729; Asking this server to re-register.
I20260501 13:53:10.122885 1116 heartbeater.cc:461] Registering TS with master...
I20260501 13:53:10.124310 2103 ts_manager.cc:194] Registered new tserver with Master: e4a8a400d10343718a43d5fcc669dd80 (127.0.105.1:44349)
I20260501 13:53:10.244988 2171 raft_consensus.cc:493] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:10.245249 2171 raft_consensus.cc:515] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:10.246994 2171 leader_election.cc:290] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5fa23b4772a944c0b0279eabd7476196 (127.0.105.62:44705), 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209)
I20260501 13:53:10.247700 1976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" candidate_term: 7 candidate_status { last_received { term: 6 index: 29 } } ignore_live_leader: false dest_uuid: "5fa23b4772a944c0b0279eabd7476196" is_pre_election: true
W20260501 13:53:10.248046 2089 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
I20260501 13:53:10.248106 1976 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e47102ddbd4f41c1bf0a2864d97237b7 in term 6.
I20260501 13:53:10.248768 2086 leader_election.cc:304] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196, e47102ddbd4f41c1bf0a2864d97237b7; no voters:
I20260501 13:53:10.249155 2171 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260501 13:53:10.249306 2171 raft_consensus.cc:493] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260501 13:53:10.249445 2171 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 6 FOLLOWER]: Advancing to term 7
W20260501 13:53:10.249557 2089 leader_election.cc:336] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
I20260501 13:53:10.252816 2171 raft_consensus.cc:515] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:10.253430 2171 leader_election.cc:290] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [CANDIDATE]: Term 7 election: Requested vote from peers 5fa23b4772a944c0b0279eabd7476196 (127.0.105.62:44705), 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209)
I20260501 13:53:10.253893 1976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" candidate_term: 7 candidate_status { last_received { term: 6 index: 29 } } ignore_live_leader: false dest_uuid: "5fa23b4772a944c0b0279eabd7476196"
I20260501 13:53:10.254149 1976 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 6 FOLLOWER]: Advancing to term 7
W20260501 13:53:10.254927 2089 leader_election.cc:336] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
I20260501 13:53:10.258030 1976 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47102ddbd4f41c1bf0a2864d97237b7 in term 7.
I20260501 13:53:10.258558 2086 leader_election.cc:304] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196, e47102ddbd4f41c1bf0a2864d97237b7; no voters: 84a39a5f167243ff9bfe07189aa881ca
I20260501 13:53:10.258953 2171 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 7 FOLLOWER]: Leader election won for term 7
I20260501 13:53:10.259454 2171 raft_consensus.cc:697] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [term 7 LEADER]: Becoming Leader. State: Replica: e47102ddbd4f41c1bf0a2864d97237b7, State: Running, Role: LEADER
I20260501 13:53:10.259963 2171 consensus_queue.cc:237] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 6.29, Last appended by leader: 29, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:10.262820 2174 sys_catalog.cc:455] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e47102ddbd4f41c1bf0a2864d97237b7. Latest consensus state: current_term: 7 leader_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:53:10.263164 2174 sys_catalog.cc:458] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [sys.catalog]: This master's current role is: LEADER
I20260501 13:53:10.653043 1976 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Refusing update from remote peer e47102ddbd4f41c1bf0a2864d97237b7: Log matching property violated. Preceding OpId in replica: term: 6 index: 28. Preceding OpId from leader: term: 7 index: 30. (index mismatch)
I20260501 13:53:10.653990 2174 consensus_queue.cc:1048] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 25, Time since last communication: 0.000s
I20260501 13:53:10.659066 2178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e47102ddbd4f41c1bf0a2864d97237b7. Latest consensus state: current_term: 7 leader_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:53:10.659417 2178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: This master's current role is: FOLLOWER
I20260501 13:53:10.663942 2174 sys_catalog.cc:455] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:53:10.664201 2174 sys_catalog.cc:458] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [sys.catalog]: This master's current role is: LEADER
I20260501 13:53:10.664539 2171 sys_catalog.cc:455] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 7 leader_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:53:10.664836 2171 sys_catalog.cc:458] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 [sys.catalog]: This master's current role is: LEADER
W20260501 13:53:10.666083 2089 consensus_peers.cc:597] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7 -> Peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Couldn't send request to peer 84a39a5f167243ff9bfe07189aa881ca. Status: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260501 13:53:10.666839 2178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:53:10.666790 2188 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:53:10.667111 2178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: This master's current role is: FOLLOWER
I20260501 13:53:10.667258 2188 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: This master's current role is: FOLLOWER
I20260501 13:53:10.677630 2176 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260501 13:53:10.680291 2176 catalog_manager.cc:679] Loaded metadata for table default.table_293c891afe6c4d609f8069858909d21b [id=6d282d8a3f814452977f46b6054268e2]
I20260501 13:53:10.680778 2176 catalog_manager.cc:679] Loaded metadata for table default.table_5a0a455cca3c4ab6a059aac962b7cea0 [id=8818494b487941a4877edf5b079d9756]
I20260501 13:53:10.681144 2176 catalog_manager.cc:679] Loaded metadata for table default.table_d7ed2811199d451c80270ccf0c6c80f4 [id=ac3b885b81d744bfaa69439967b2785f]
I20260501 13:53:10.681478 2176 catalog_manager.cc:679] Loaded metadata for table default.table_5a6d83f7292440209933a379074aadbb [id=b0561cbad3a94637a54b01f351d94fbb]
I20260501 13:53:10.681908 2176 catalog_manager.cc:679] Loaded metadata for table default.table_06eba6fa1b4949ffa139c461e70349f9 [id=ded9bef73cb942de81c7b635d42480f8]
I20260501 13:53:10.682255 2176 catalog_manager.cc:679] Loaded metadata for table default.table_8beb55fb2bdd42309d0183432704b989 [id=f7226f45b9d94af2ad176c2d4a419e88]
I20260501 13:53:10.684237 2176 tablet_loader.cc:96] loaded metadata for tablet 31bef174b83549ad95990b0ddb64360f (table default.table_293c891afe6c4d609f8069858909d21b [id=6d282d8a3f814452977f46b6054268e2])
I20260501 13:53:10.685015 2176 tablet_loader.cc:96] loaded metadata for tablet 5ac1be5c8f924184956792292483b347 (table default.table_d7ed2811199d451c80270ccf0c6c80f4 [id=ac3b885b81d744bfaa69439967b2785f])
I20260501 13:53:10.685832 2176 tablet_loader.cc:96] loaded metadata for tablet b1231946d86d49caa3c9fcb632b38896 (table default.table_8beb55fb2bdd42309d0183432704b989 [id=f7226f45b9d94af2ad176c2d4a419e88])
I20260501 13:53:10.686117 2176 tablet_loader.cc:96] loaded metadata for tablet bcf6cd634f444b1d90c0440134510d1e (table default.table_5a6d83f7292440209933a379074aadbb [id=b0561cbad3a94637a54b01f351d94fbb])
I20260501 13:53:10.686524 2176 tablet_loader.cc:96] loaded metadata for tablet cf800b1230eb4b5eac7feb0276d54486 (table default.table_06eba6fa1b4949ffa139c461e70349f9 [id=ded9bef73cb942de81c7b635d42480f8])
I20260501 13:53:10.686812 2176 tablet_loader.cc:96] loaded metadata for tablet daa28e4cd0d042b5bb7f7ea04ddb19b7 (table default.table_5a0a455cca3c4ab6a059aac962b7cea0 [id=8818494b487941a4877edf5b079d9756])
I20260501 13:53:10.687110 2176 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260501 13:53:10.688135 2176 catalog_manager.cc:1269] Loaded cluster ID: 6424d308ce1d424b8fe7d2c1b812f4f7
I20260501 13:53:10.688238 2176 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260501 13:53:10.689496 2176 catalog_manager.cc:1514] Loading token signing keys...
I20260501 13:53:10.690322 2176 catalog_manager.cc:6055] T 00000000000000000000000000000000 P e47102ddbd4f41c1bf0a2864d97237b7: Loaded TSK: 0
I20260501 13:53:10.691210 2176 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260501 13:53:10.692159 2176 catalog_manager.cc:1537] Loading latest processed Hive Metastore notification log event ID...
I20260501 13:53:10.693063 2176 catalog_manager.cc:5893] Last processed Hive Metastore notification event ID: -1
F20260501 13:53:10.707962 2157 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 11 received events: NotificationEvent(eventId=1, eventTime=1777643577, eventType=CREATE_TABLE, dbName=default, tableName=table_5a0a455cca3c4ab6a059aac962b7cea0, message={"server":"","servicePrincipal":"","db":"default","table":"table_5a0a455cca3c4ab6a059aac962b7cea0","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_5a0a455cca3c4ab6a059aac962b7cea0\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1777643577},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/warehouse/table_5a0a455cca3c4ab6a059aac962b7cea0\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_5a0a455cca3c4ab6a059aac962b7cea0\",\"kudu.table_id\":\"8818494b487941a4877edf5b079d9756\",\"transient_lastDdlTime\":\"1777643577\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"6424d308ce1d424b8fe7d2c1b812f4f7\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1777643577,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1777643577, eventType=CREATE_TABLE, dbName=default, tableName=table_86cdd461d6044be8b0ae522dfcc0714a, message={"server":"","servicePrincipal":"","db":"default","table":"table_86cdd461d6044be8b0ae522dfcc0714a","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_86cdd461d6044be8b0ae522dfcc0714a\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1777643577},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/warehouse/table_86cdd461d6044be8b0ae522dfcc0714a\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_86cdd461d6044be8b0ae522dfcc0714a\",\"kudu.table_id\":\"df2e28d52fd341ffac31641fe0482fe8\",\"transient_lastDdlTime\":\"1777643577\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"6424d308ce1d424b8fe7d2c1b812f4f7\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1777643577,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
@ 0x7ff993da9dcd google::LogMessage::Fail() at ??:0
@ 0x7ff993dadb93 google::LogMessage::SendToLog() at ??:0
@ 0x7ff993da97cc google::LogMessage::Flush() at ??:0
@ 0x7ff993da9a11 google::LogMessage::~LogMessage() at ??:0
@ 0x7ff9aa862574 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7ff9aa860e21 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7ff9aa868a9c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7ff9aa868919 std::_Function_handler<>::_M_invoke() at ??:0
I20260501 13:53:10.754549 1051 tablet_service.cc:1511] Processing CreateTablet for tablet b1231946d86d49caa3c9fcb632b38896 (DEFAULT_TABLE table=default.table_8beb55fb2bdd42309d0183432704b989 [id=f7226f45b9d94af2ad176c2d4a419e88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260501 13:53:10.754830 1049 tablet_service.cc:1511] Processing CreateTablet for tablet cf800b1230eb4b5eac7feb0276d54486 (DEFAULT_TABLE table=default.table_06eba6fa1b4949ffa139c461e70349f9 [id=ded9bef73cb942de81c7b635d42480f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260501 13:53:10.755081 1051 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1231946d86d49caa3c9fcb632b38896. 1 dirs total, 0 dirs full, 0 dirs failed
I20260501 13:53:10.755352 1049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf800b1230eb4b5eac7feb0276d54486. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0xaa4cad std::function<>::operator()() at ??:?
I20260501 13:53:10.760987 1050 tablet_service.cc:1511] Processing CreateTablet for tablet bcf6cd634f444b1d90c0440134510d1e (DEFAULT_TABLE table=default.table_5a6d83f7292440209933a379074aadbb [id=b0561cbad3a94637a54b01f351d94fbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260501 13:53:10.761466 1050 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcf6cd634f444b1d90c0440134510d1e. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7ff996b42529 kudu::Thread::SuperviseThread() at ??:0
I20260501 13:53:10.764339 1048 tablet_service.cc:1511] Processing CreateTablet for tablet 31bef174b83549ad95990b0ddb64360f (DEFAULT_TABLE table=default.table_293c891afe6c4d609f8069858909d21b [id=6d282d8a3f814452977f46b6054268e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260501 13:53:10.764848 1048 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31bef174b83549ad95990b0ddb64360f. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7ff99a1106db start_thread at ??:0
@ 0x7ff992c5a61f clone at ??:0
I20260501 13:53:10.775132 2205 tablet_bootstrap.cc:492] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80: Bootstrap starting.
I20260501 13:53:10.777030 2205 tablet_bootstrap.cc:654] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80: Neither blocks nor log segments found. Creating new log.
I20260501 13:53:10.778923 2205 tablet_bootstrap.cc:492] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80: No bootstrap required, opened a new log
I20260501 13:53:10.779093 2205 ts_tablet_manager.cc:1403] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260501 13:53:10.780025 2205 raft_consensus.cc:359] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:53:10.780237 2205 raft_consensus.cc:385] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260501 13:53:10.780323 2205 raft_consensus.cc:740] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4a8a400d10343718a43d5fcc669dd80, State: Initialized, Role: FOLLOWER
I20260501 13:53:10.780566 2205 consensus_queue.cc:260] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:53:10.782004 2205 ts_tablet_manager.cc:1434] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260501 13:53:10.782310 2205 tablet_bootstrap.cc:492] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80: Bootstrap starting.
I20260501 13:53:10.784674 2205 tablet_bootstrap.cc:654] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80: Neither blocks nor log segments found. Creating new log.
I20260501 13:53:10.786654 2205 tablet_bootstrap.cc:492] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80: No bootstrap required, opened a new log
I20260501 13:53:10.786780 2205 ts_tablet_manager.cc:1403] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.002s
I20260501 13:53:10.787357 2205 raft_consensus.cc:359] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } }
I20260501 13:53:10.787504 2205 raft_consensus.cc:385] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260501 13:53:10.787577 2205 raft_consensus.cc:740] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4a8a400d10343718a43d5fcc669dd80, State: Initialized, Role: FOLLOWER
I20260501 13:53:10.787752 2205 consensus_queue.cc:260] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } }
I20260501 13:53:10.788182 2205 ts_tablet_manager.cc:1434] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260501 13:53:10.789979 2205 tablet_bootstrap.cc:492] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80: Bootstrap starting.
I20260501 13:53:10.792464 2205 tablet_bootstrap.cc:654] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80: Neither blocks nor log segments found. Creating new log.
I20260501 13:53:10.794042 2205 tablet_bootstrap.cc:492] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80: No bootstrap required, opened a new log
I20260501 13:53:10.794143 2205 ts_tablet_manager.cc:1403] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260501 13:53:10.794691 2205 raft_consensus.cc:359] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:53:10.794823 2205 raft_consensus.cc:385] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260501 13:53:10.794881 2205 raft_consensus.cc:740] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4a8a400d10343718a43d5fcc669dd80, State: Initialized, Role: FOLLOWER
I20260501 13:53:10.795132 2205 consensus_queue.cc:260] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:53:10.796298 2205 ts_tablet_manager.cc:1434] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260501 13:53:10.796619 2205 tablet_bootstrap.cc:492] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80: Bootstrap starting.
I20260501 13:53:10.798904 2205 tablet_bootstrap.cc:654] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80: Neither blocks nor log segments found. Creating new log.
I20260501 13:53:10.801425 2205 tablet_bootstrap.cc:492] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80: No bootstrap required, opened a new log
I20260501 13:53:10.801591 2205 ts_tablet_manager.cc:1403] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260501 13:53:10.802428 2205 raft_consensus.cc:359] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:53:10.802636 2205 raft_consensus.cc:385] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260501 13:53:10.802724 2205 raft_consensus.cc:740] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4a8a400d10343718a43d5fcc669dd80, State: Initialized, Role: FOLLOWER
I20260501 13:53:10.802982 2205 consensus_queue.cc:260] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:53:10.803550 2205 ts_tablet_manager.cc:1434] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260501 13:53:10.945752 1006 connection.cc:570] server connection from 127.0.0.1:43664 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260501 13:53:10.946218 2197 negotiation.cc:336] Failed RPC negotiation. Trace:
0501 13:53:10.732883 (+ 0us) reactor.cc:678] Submitting negotiation task for server connection from 127.0.0.1:59208 (local address 127.0.105.2:43565)
0501 13:53:10.757937 (+ 25054us) server_negotiation.cc:207] Beginning negotiation
0501 13:53:10.757944 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0501 13:53:10.757968 (+ 24us) server_negotiation.cc:408] Connection header received
0501 13:53:10.758052 (+ 84us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0501 13:53:10.758058 (+ 6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0501 13:53:10.758140 (+ 82us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0501 13:53:10.758277 (+ 137us) server_negotiation.cc:227] Negotiated authn=CERTIFICATE
0501 13:53:10.761119 (+ 2842us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0501 13:53:10.762582 (+ 1463us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0501 13:53:10.945828 (+183246us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59208: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":24908,"thread_start_us":25580,"threads_started":1}
W20260501 13:53:10.946841 1004 connection.cc:570] client connection to 127.0.105.60:39729 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260501 13:53:10.946970 2202 negotiation.cc:336] Failed RPC negotiation. Trace:
0501 13:53:10.760871 (+ 0us) reactor.cc:678] Submitting negotiation task for server connection from 127.0.0.1:57142 (local address 127.0.105.3:41841)
0501 13:53:10.765879 (+ 5008us) server_negotiation.cc:207] Beginning negotiation
0501 13:53:10.765887 (+ 8us) server_negotiation.cc:400] Waiting for connection header
0501 13:53:10.765910 (+ 23us) server_negotiation.cc:408] Connection header received
0501 13:53:10.765990 (+ 80us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0501 13:53:10.765996 (+ 6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0501 13:53:10.766085 (+ 89us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0501 13:53:10.766225 (+ 140us) server_negotiation.cc:227] Negotiated authn=CERTIFICATE
0501 13:53:10.768976 (+ 2751us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0501 13:53:10.770587 (+ 1611us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0501 13:53:10.945873 (+175286us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57142: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":4871,"thread_start_us":5512,"threads_started":1}
I20260501 13:53:10.954195 2168 raft_consensus.cc:493] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:10.954396 2168 raft_consensus.cc:515] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:53:10.955135 2168 leader_election.cc:290] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:53:10.956365 1007 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
W20260501 13:53:10.956631 1005 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
I20260501 13:53:10.956770 1005 leader_election.cc:304] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:53:10.957019 2168 raft_consensus.cc:2749] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:53:11.052376 2168 raft_consensus.cc:493] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:11.052632 2168 raft_consensus.cc:515] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } }
I20260501 13:53:11.053293 2168 leader_election.cc:290] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:53:11.054400 1007 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
W20260501 13:53:11.054589 1005 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
I20260501 13:53:11.054706 1005 leader_election.cc:304] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:53:11.054927 2168 raft_consensus.cc:2749] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:53:11.222043 1931 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:53:11.222270 1931 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:53:11.230310 2215 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e47102ddbd4f41c1bf0a2864d97237b7)
I20260501 13:53:11.230530 2215 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:11.231302 2215 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:53:11.233414 1951 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:53:11.233582 1948 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:53:11.233709 1948 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:53:11.233952 2215 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:53:11.247370 1388 heartbeater.cc:499] Master 127.0.105.60:39729 was elected leader, sending a full tablet report...
I20260501 13:53:11.653726 2168 raft_consensus.cc:493] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:11.653991 2168 raft_consensus.cc:515] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:53:11.654804 2168 leader_election.cc:290] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:53:11.655706 1005 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
W20260501 13:53:11.655923 1007 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
I20260501 13:53:11.656075 1007 leader_election.cc:304] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:53:11.656345 2168 raft_consensus.cc:2749] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:53:11.695120 2168 raft_consensus.cc:493] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:11.695415 2168 raft_consensus.cc:515] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:53:11.696167 2168 leader_election.cc:290] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:53:11.697074 1007 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
W20260501 13:53:11.697314 1005 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
I20260501 13:53:11.697453 1005 leader_election.cc:304] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:53:11.697711 2168 raft_consensus.cc:2749] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:53:11.935595 2219 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e47102ddbd4f41c1bf0a2864d97237b7)
I20260501 13:53:11.935837 2219 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:11.936709 2219 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:53:11.937839 1948 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111) [suppressed 9 similar messages]
W20260501 13:53:11.939312 1948 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
W20260501 13:53:11.939509 1951 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
I20260501 13:53:11.939751 1951 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:53:11.940001 2219 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:53:12.143204 420 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-task4ezAfy/build/asan/bin/kudu pid 1931
I20260501 13:53:12.149916 1996 generic_service.cc:196] Checking for leaks (request via RPC)
I20260501 13:53:12.259773 420 external_mini_cluster.cc:1658] Killing /tmp/dist-test-task4ezAfy/build/asan/bin/kudu with pid 1931
I20260501 13:53:12.920956 420 external_mini_cluster.cc:1366] Running /tmp/dist-test-task4ezAfy/build/asan/bin/kudu
/tmp/dist-test-task4ezAfy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.62:44705
--webserver_interface=127.0.105.62
--webserver_port=45017
--builtin_ntp_servers=127.0.105.20:38455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729
--hive_metastore_uris=thrift://127.0.0.1:33681
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260501 13:53:13.281235 2228 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260501 13:53:13.281615 2228 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260501 13:53:13.281746 2228 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260501 13:53:13.290810 2228 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260501 13:53:13.290974 2228 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260501 13:53:13.291114 2228 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260501 13:53:13.291198 2228 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260501 13:53:13.301520 2228 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.105.20:38455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:33681
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.62:44705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.105.62
--webserver_port=45017
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision aa7cc0aa9c559b87f264235eaadd03ec492277e4
build type FASTDEBUG
built by None at 01 May 2026 13:43:23 UTC on bdcb31816ec0
build id 11694
ASAN enabled
I20260501 13:53:13.303195 2228 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260501 13:53:13.305013 2228 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260501 13:53:13.313652 2235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260501 13:53:13.313675 2234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260501 13:53:13.314486 2237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260501 13:53:13.316656 2228 server_base.cc:1061] running on GCE node
I20260501 13:53:13.317533 2228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260501 13:53:13.319167 2228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260501 13:53:13.320472 2228 hybrid_clock.cc:648] HybridClock initialized: now 1777643593320422 us; error 35 us; skew 500 ppm
I20260501 13:53:13.320957 2228 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260501 13:53:13.323918 2228 webserver.cc:492] Webserver started at http://127.0.105.62:45017/ using document root <none> and password file <none>
I20260501 13:53:13.324937 2228 fs_manager.cc:362] Metadata directory not provided
I20260501 13:53:13.325047 2228 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260501 13:53:13.331430 2228 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260501 13:53:13.334872 2243 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260501 13:53:13.336541 2228 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260501 13:53:13.336771 2228 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data,/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/wal
uuid: "5fa23b4772a944c0b0279eabd7476196"
format_stamp: "Formatted at 2026-05-01 13:52:53 on dist-test-slave-qlwv"
I20260501 13:53:13.337536 2228 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260501 13:53:13.358029 2228 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260501 13:53:13.358989 2228 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260501 13:53:13.359368 2228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260501 13:53:13.381556 2228 rpc_server.cc:307] RPC server started. Bound to: 127.0.105.62:44705
I20260501 13:53:13.381599 2295 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.105.62:44705 every 8 connection(s)
I20260501 13:53:13.383407 2228 server_base.cc:1193] Dumped server information to /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data/info.pb
I20260501 13:53:13.390318 2296 sys_catalog.cc:263] Verifying existing consensus state
I20260501 13:53:13.391628 420 external_mini_cluster.cc:1428] Started /tmp/dist-test-task4ezAfy/build/asan/bin/kudu as pid 2228
I20260501 13:53:13.394428 2296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: Bootstrap starting.
I20260501 13:53:13.408242 2296 log.cc:826] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: Log is configured to *not* fsync() on all Append() calls
I20260501 13:53:13.434170 2296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: Bootstrap replayed 1/1 log segments. Stats: ops{read=31 overwritten=0 applied=31 ignored=0} inserts{seen=23 ignored=0} mutations{seen=23 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260501 13:53:13.434777 2296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: Bootstrap complete.
I20260501 13:53:13.440384 2296 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:13.441133 2296 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fa23b4772a944c0b0279eabd7476196, State: Initialized, Role: FOLLOWER
I20260501 13:53:13.441761 2296 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 31, Last appended: 7.31, Last appended by leader: 31, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:13.442543 2300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 7 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:53:13.442925 2300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: This master's current role is: FOLLOWER
I20260501 13:53:13.444056 2296 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: configured and running, proceeding with master startup.
I20260501 13:53:13.468356 2314 catalog_manager.cc:1269] Loaded cluster ID: 6424d308ce1d424b8fe7d2c1b812f4f7
I20260501 13:53:13.468513 2314 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: loading cluster ID for follower catalog manager: success
I20260501 13:53:13.473637 2314 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: acquiring CA information for follower catalog manager: success
I20260501 13:53:13.477020 2314 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260501 13:53:13.478653 2228 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:53:13.484753 2228 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:53:13.550297 1255 heartbeater.cc:344] Connected to a master server at 127.0.105.62:44705
I20260501 13:53:13.553473 2260 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" instance_seqno: 1777643576483530) as {username='slave'} at 127.0.105.2:47081; Asking this server to re-register.
I20260501 13:53:13.554275 1255 heartbeater.cc:461] Registering TS with master...
I20260501 13:53:13.556322 2260 ts_manager.cc:194] Registered new tserver with Master: 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
I20260501 13:53:13.769973 1389 heartbeater.cc:344] Connected to a master server at 127.0.105.62:44705
I20260501 13:53:13.771538 2260 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" instance_seqno: 1777643576999180) as {username='slave'} at 127.0.105.3:59087; Asking this server to re-register.
I20260501 13:53:13.772212 1389 heartbeater.cc:461] Registering TS with master...
I20260501 13:53:13.773839 2260 ts_manager.cc:194] Registered new tserver with Master: b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841)
I20260501 13:53:13.821213 1117 heartbeater.cc:344] Connected to a master server at 127.0.105.62:44705
I20260501 13:53:13.822692 2260 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" instance_seqno: 1777643575975284) as {username='slave'} at 127.0.105.1:42431; Asking this server to re-register.
I20260501 13:53:13.823467 1117 heartbeater.cc:461] Registering TS with master...
I20260501 13:53:13.825060 2260 ts_manager.cc:194] Registered new tserver with Master: e4a8a400d10343718a43d5fcc669dd80 (127.0.105.1:44349)
I20260501 13:53:13.954921 2320 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:13.955353 2320 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:13.958181 2320 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:53:13.959301 2247 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:53:13.961422 2247 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:53:13.961697 2244 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:53:13.961911 2244 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:53:13.962321 2320 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:53:14.594755 2325 raft_consensus.cc:493] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:14.594988 2325 raft_consensus.cc:515] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } }
I20260501 13:53:14.595911 2325 leader_election.cc:290] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:53:14.596975 1007 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
W20260501 13:53:14.597234 1005 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
I20260501 13:53:14.597379 1005 leader_election.cc:304] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:53:14.597644 2325 raft_consensus.cc:2749] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:53:14.693413 2326 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:14.693667 2326 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:14.694614 2326 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:53:14.696636 2247 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:53:14.696923 2244 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:53:14.697059 2244 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:53:14.697314 2326 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:53:14.898509 2325 raft_consensus.cc:493] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:14.898782 2325 raft_consensus.cc:515] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:53:14.899582 2325 leader_election.cc:290] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:53:14.900429 1005 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
W20260501 13:53:14.900563 1007 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
I20260501 13:53:14.900661 1005 leader_election.cc:304] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:53:14.900913 2325 raft_consensus.cc:2749] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:53:15.445063 2331 raft_consensus.cc:493] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:15.445320 2331 raft_consensus.cc:515] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:53:15.446131 2331 leader_election.cc:290] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:53:15.447105 1005 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
W20260501 13:53:15.447325 1007 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
I20260501 13:53:15.447480 1007 leader_election.cc:304] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:53:15.447734 2331 raft_consensus.cc:2749] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:53:15.468555 2332 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:15.468757 2332 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:15.469477 2332 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:53:15.471766 2244 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
W20260501 13:53:15.472011 2247 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
I20260501 13:53:15.472230 2247 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:53:15.472517 2332 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:53:15.642444 2331 raft_consensus.cc:493] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:15.642671 2331 raft_consensus.cc:515] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:53:15.643492 2331 leader_election.cc:290] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:53:15.644354 1005 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
W20260501 13:53:15.644515 1007 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
I20260501 13:53:15.644667 1007 leader_election.cc:304] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:53:15.645120 2331 raft_consensus.cc:2749] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:53:15.829748 420 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-task4ezAfy/build/asan/bin/kudu pid 2228
I20260501 13:53:15.846482 2291 generic_service.cc:196] Checking for leaks (request via RPC)
I20260501 13:53:15.958863 420 external_mini_cluster.cc:1658] Killing /tmp/dist-test-task4ezAfy/build/asan/bin/kudu with pid 2228
I20260501 13:53:16.757263 420 external_mini_cluster.cc:1366] Running /tmp/dist-test-task4ezAfy/build/asan/bin/kudu
/tmp/dist-test-task4ezAfy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.62:44705
--webserver_interface=127.0.105.62
--webserver_port=45017
--builtin_ntp_servers=127.0.105.20:38455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729
--hive_metastore_uris=thrift://127.0.0.1:33681
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260501 13:53:17.116021 2342 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260501 13:53:17.116391 2342 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260501 13:53:17.116518 2342 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260501 13:53:17.125612 2342 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260501 13:53:17.125757 2342 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260501 13:53:17.125885 2342 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260501 13:53:17.125972 2342 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260501 13:53:17.136519 2342 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.105.20:38455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:33681
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.62:44705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.105.62
--webserver_port=45017
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision aa7cc0aa9c559b87f264235eaadd03ec492277e4
build type FASTDEBUG
built by None at 01 May 2026 13:43:23 UTC on bdcb31816ec0
build id 11694
ASAN enabled
I20260501 13:53:17.138258 2342 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260501 13:53:17.140244 2342 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260501 13:53:17.148514 2348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260501 13:53:17.148488 2349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260501 13:53:17.149189 2351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260501 13:53:17.149503 2342 server_base.cc:1061] running on GCE node
I20260501 13:53:17.150257 2342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260501 13:53:17.151688 2342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260501 13:53:17.152920 2342 hybrid_clock.cc:648] HybridClock initialized: now 1777643597152843 us; error 49 us; skew 500 ppm
I20260501 13:53:17.153390 2342 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260501 13:53:17.155807 2342 webserver.cc:492] Webserver started at http://127.0.105.62:45017/ using document root <none> and password file <none>
I20260501 13:53:17.156703 2342 fs_manager.cc:362] Metadata directory not provided
I20260501 13:53:17.156870 2342 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260501 13:53:17.162806 2342 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260501 13:53:17.166052 2357 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260501 13:53:17.167582 2342 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260501 13:53:17.167776 2342 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data,/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/wal
uuid: "5fa23b4772a944c0b0279eabd7476196"
format_stamp: "Formatted at 2026-05-01 13:52:53 on dist-test-slave-qlwv"
I20260501 13:53:17.168501 2342 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260501 13:53:17.182580 2342 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260501 13:53:17.183560 2342 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260501 13:53:17.183874 2342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260501 13:53:17.203868 2342 rpc_server.cc:307] RPC server started. Bound to: 127.0.105.62:44705
I20260501 13:53:17.203889 2408 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.105.62:44705 every 8 connection(s)
I20260501 13:53:17.205514 2342 server_base.cc:1193] Dumped server information to /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data/info.pb
I20260501 13:53:17.207298 420 external_mini_cluster.cc:1428] Started /tmp/dist-test-task4ezAfy/build/asan/bin/kudu as pid 2342
I20260501 13:53:17.213071 2409 sys_catalog.cc:263] Verifying existing consensus state
I20260501 13:53:17.216405 2409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: Bootstrap starting.
I20260501 13:53:17.227015 2409 log.cc:826] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: Log is configured to *not* fsync() on all Append() calls
I20260501 13:53:17.251791 2409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: Bootstrap replayed 1/1 log segments. Stats: ops{read=31 overwritten=0 applied=31 ignored=0} inserts{seen=23 ignored=0} mutations{seen=23 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260501 13:53:17.252380 2409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: Bootstrap complete.
I20260501 13:53:17.257599 2409 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:17.258304 2409 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fa23b4772a944c0b0279eabd7476196, State: Initialized, Role: FOLLOWER
I20260501 13:53:17.259038 2409 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 31, Last appended: 7.31, Last appended by leader: 31, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:17.259830 2414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 7 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:53:17.260074 2414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: This master's current role is: FOLLOWER
I20260501 13:53:17.260898 2409 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: configured and running, proceeding with master startup.
I20260501 13:53:17.288136 2428 catalog_manager.cc:1269] Loaded cluster ID: 6424d308ce1d424b8fe7d2c1b812f4f7
I20260501 13:53:17.288365 2428 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: loading cluster ID for follower catalog manager: success
I20260501 13:53:17.282447 2342 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:53:17.289734 2342 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:53:17.297392 1389 heartbeater.cc:344] Connected to a master server at 127.0.105.62:44705
I20260501 13:53:17.298408 2428 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: acquiring CA information for follower catalog manager: success
I20260501 13:53:17.300310 2374 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" instance_seqno: 1777643576999180) as {username='slave'} at 127.0.105.3:52823; Asking this server to re-register.
I20260501 13:53:17.301024 1389 heartbeater.cc:461] Registering TS with master...
I20260501 13:53:17.302673 2428 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260501 13:53:17.302969 2374 ts_manager.cc:194] Registered new tserver with Master: b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841)
I20260501 13:53:17.345460 1117 heartbeater.cc:344] Connected to a master server at 127.0.105.62:44705
I20260501 13:53:17.346886 2374 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" instance_seqno: 1777643575975284) as {username='slave'} at 127.0.105.1:36761; Asking this server to re-register.
I20260501 13:53:17.347538 1117 heartbeater.cc:461] Registering TS with master...
I20260501 13:53:17.348860 2374 ts_manager.cc:194] Registered new tserver with Master: e4a8a400d10343718a43d5fcc669dd80 (127.0.105.1:44349)
I20260501 13:53:17.577337 1255 heartbeater.cc:344] Connected to a master server at 127.0.105.62:44705
I20260501 13:53:17.578811 2374 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" instance_seqno: 1777643576483530) as {username='slave'} at 127.0.105.2:39697; Asking this server to re-register.
I20260501 13:53:17.579495 1255 heartbeater.cc:461] Registering TS with master...
I20260501 13:53:17.580829 2374 ts_manager.cc:194] Registered new tserver with Master: 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
I20260501 13:53:17.701575 2414 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:17.701886 2414 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:17.703891 2414 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:53:17.704608 2358 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
W20260501 13:53:17.706146 2358 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
W20260501 13:53:17.706377 2361 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
I20260501 13:53:17.706503 2361 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:53:17.706815 2414 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:53:18.280230 2436 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:18.280433 2436 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:18.281121 2436 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:53:18.283108 2358 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
W20260501 13:53:18.283320 2361 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
I20260501 13:53:18.283422 2361 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:53:18.283660 2436 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:53:18.422549 2435 raft_consensus.cc:493] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:18.422794 2435 raft_consensus.cc:515] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } }
I20260501 13:53:18.423729 2435 leader_election.cc:290] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:53:18.424732 1005 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
W20260501 13:53:18.424930 1007 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
I20260501 13:53:18.425063 1007 leader_election.cc:304] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:53:18.425400 2435 raft_consensus.cc:2749] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:53:19.392166 2442 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:19.392344 2442 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:19.392930 2442 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:53:19.397156 2358 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
W20260501 13:53:19.397361 2361 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
I20260501 13:53:19.397475 2361 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:53:19.397739 2442 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:53:19.742566 420 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-task4ezAfy/build/asan/bin/kudu pid 2342
I20260501 13:53:19.749807 2404 generic_service.cc:196] Checking for leaks (request via RPC)
I20260501 13:53:19.859416 420 external_mini_cluster.cc:1658] Killing /tmp/dist-test-task4ezAfy/build/asan/bin/kudu with pid 2342
I20260501 13:53:20.391861 420 external_mini_cluster.cc:1366] Running /tmp/dist-test-task4ezAfy/build/asan/bin/kudu
/tmp/dist-test-task4ezAfy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.62:44705
--webserver_interface=127.0.105.62
--webserver_port=45017
--builtin_ntp_servers=127.0.105.20:38455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729
--hive_metastore_uris=thrift://127.0.0.1:33681
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260501 13:53:20.417291 2449 raft_consensus.cc:493] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:20.417479 2449 raft_consensus.cc:515] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:53:20.418241 2449 leader_election.cc:290] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:53:20.419188 1007 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
W20260501 13:53:20.419404 1005 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
I20260501 13:53:20.419554 1005 leader_election.cc:304] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:53:20.419817 2449 raft_consensus.cc:2749] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:53:20.558179 2449 raft_consensus.cc:493] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:20.558404 2449 raft_consensus.cc:515] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:53:20.559120 2449 leader_election.cc:290] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:53:20.559991 1007 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
W20260501 13:53:20.560155 1005 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
I20260501 13:53:20.560343 1005 leader_election.cc:304] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:53:20.560621 2449 raft_consensus.cc:2749] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260501 13:53:20.728641 2451 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260501 13:53:20.728976 2451 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260501 13:53:20.729048 2451 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260501 13:53:20.738721 2451 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260501 13:53:20.738818 2451 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260501 13:53:20.738880 2451 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260501 13:53:20.738982 2451 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260501 13:53:20.750134 2451 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.105.20:38455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:33681
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.62:44705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.105.62
--webserver_port=45017
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision aa7cc0aa9c559b87f264235eaadd03ec492277e4
build type FASTDEBUG
built by None at 01 May 2026 13:43:23 UTC on bdcb31816ec0
build id 11694
ASAN enabled
I20260501 13:53:20.751811 2451 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260501 13:53:20.753540 2451 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260501 13:53:20.762507 2458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260501 13:53:20.762593 2457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260501 13:53:20.763968 2451 server_base.cc:1061] running on GCE node
W20260501 13:53:20.765005 2460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260501 13:53:20.765815 2451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260501 13:53:20.767400 2451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260501 13:53:20.768625 2451 hybrid_clock.cc:648] HybridClock initialized: now 1777643600768580 us; error 79 us; skew 500 ppm
I20260501 13:53:20.768971 2451 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260501 13:53:20.771327 2451 webserver.cc:492] Webserver started at http://127.0.105.62:45017/ using document root <none> and password file <none>
I20260501 13:53:20.772071 2451 fs_manager.cc:362] Metadata directory not provided
I20260501 13:53:20.772217 2451 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260501 13:53:20.777797 2451 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.001s
I20260501 13:53:20.780889 2466 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260501 13:53:20.782298 2451 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260501 13:53:20.782490 2451 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data,/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/wal
uuid: "5fa23b4772a944c0b0279eabd7476196"
format_stamp: "Formatted at 2026-05-01 13:52:53 on dist-test-slave-qlwv"
I20260501 13:53:20.783197 2451 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260501 13:53:20.801427 2451 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260501 13:53:20.802281 2451 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260501 13:53:20.802569 2451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260501 13:53:20.822741 2451 rpc_server.cc:307] RPC server started. Bound to: 127.0.105.62:44705
I20260501 13:53:20.822757 2517 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.105.62:44705 every 8 connection(s)
I20260501 13:53:20.824465 2451 server_base.cc:1193] Dumped server information to /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data/info.pb
I20260501 13:53:20.831516 420 external_mini_cluster.cc:1428] Started /tmp/dist-test-task4ezAfy/build/asan/bin/kudu as pid 2451
I20260501 13:53:20.831550 2518 sys_catalog.cc:263] Verifying existing consensus state
I20260501 13:53:20.835587 2518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: Bootstrap starting.
I20260501 13:53:20.850883 2518 log.cc:826] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: Log is configured to *not* fsync() on all Append() calls
I20260501 13:53:20.867594 1389 heartbeater.cc:344] Connected to a master server at 127.0.105.62:44705
I20260501 13:53:20.876477 2518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: Bootstrap replayed 1/1 log segments. Stats: ops{read=31 overwritten=0 applied=31 ignored=0} inserts{seen=23 ignored=0} mutations{seen=23 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260501 13:53:20.877063 2518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: Bootstrap complete.
I20260501 13:53:20.882622 2518 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:20.883428 2518 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fa23b4772a944c0b0279eabd7476196, State: Initialized, Role: FOLLOWER
I20260501 13:53:20.884114 2518 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 31, Last appended: 7.31, Last appended by leader: 31, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:20.885035 2523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 7 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:53:20.885318 2523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: This master's current role is: FOLLOWER
I20260501 13:53:20.886317 2518 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: configured and running, proceeding with master startup.
I20260501 13:53:20.888154 1117 heartbeater.cc:344] Connected to a master server at 127.0.105.62:44705
I20260501 13:53:20.908844 2537 catalog_manager.cc:1269] Loaded cluster ID: 6424d308ce1d424b8fe7d2c1b812f4f7
I20260501 13:53:20.909044 2537 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: loading cluster ID for follower catalog manager: success
I20260501 13:53:20.914250 2537 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: acquiring CA information for follower catalog manager: success
I20260501 13:53:20.917445 2537 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260501 13:53:20.914227 2451 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:53:20.918128 2451 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:53:21.100807 1255 heartbeater.cc:344] Connected to a master server at 127.0.105.62:44705
I20260501 13:53:21.103092 2483 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" instance_seqno: 1777643576483530) as {username='slave'} at 127.0.105.2:42255; Asking this server to re-register.
I20260501 13:53:21.103912 1255 heartbeater.cc:461] Registering TS with master...
I20260501 13:53:21.105690 2483 ts_manager.cc:194] Registered new tserver with Master: 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
I20260501 13:53:21.371441 2483 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" instance_seqno: 1777643576999180) as {username='slave'} at 127.0.105.3:56273; Asking this server to re-register.
I20260501 13:53:21.372144 1389 heartbeater.cc:461] Registering TS with master...
I20260501 13:53:21.373414 2483 ts_manager.cc:194] Registered new tserver with Master: b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841)
I20260501 13:53:21.391279 2483 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" instance_seqno: 1777643575975284) as {username='slave'} at 127.0.105.1:60483; Asking this server to re-register.
I20260501 13:53:21.391952 1117 heartbeater.cc:461] Registering TS with master...
I20260501 13:53:21.393242 2483 ts_manager.cc:194] Registered new tserver with Master: e4a8a400d10343718a43d5fcc669dd80 (127.0.105.1:44349)
I20260501 13:53:21.453940 2541 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:21.454180 2541 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:21.456017 2541 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:53:21.456919 2470 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:53:21.458349 2470 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:53:21.458582 2467 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:53:21.458726 2467 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:53:21.459096 2541 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:53:21.520401 2545 raft_consensus.cc:493] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:21.520617 2545 raft_consensus.cc:515] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:53:21.521351 2545 leader_election.cc:290] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:53:21.522238 1007 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
W20260501 13:53:21.522424 1005 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
I20260501 13:53:21.522567 1005 leader_election.cc:304] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:53:21.522831 2545 raft_consensus.cc:2749] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:53:22.005990 2547 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:22.006227 2547 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:22.007109 2547 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:53:22.008858 2470 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:53:22.009166 2467 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:53:22.009255 2467 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:53:22.009518 2547 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:53:22.783278 2553 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:22.783468 2553 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:22.784077 2553 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:53:22.786068 2470 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:53:22.786312 2467 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:53:22.786458 2467 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:53:22.786679 2553 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:53:23.215458 420 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-task4ezAfy/build/asan/bin/kudu pid 2451
I20260501 13:53:23.222476 2513 generic_service.cc:196] Checking for leaks (request via RPC)
I20260501 13:53:23.330765 420 external_mini_cluster.cc:1658] Killing /tmp/dist-test-task4ezAfy/build/asan/bin/kudu with pid 2451
I20260501 13:53:24.330519 2562 raft_consensus.cc:493] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:24.330782 2562 raft_consensus.cc:515] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } }
I20260501 13:53:24.331818 2562 leader_election.cc:290] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:53:24.332983 1007 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
W20260501 13:53:24.333199 1005 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
I20260501 13:53:24.333379 1005 leader_election.cc:304] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:53:24.333717 2562 raft_consensus.cc:2749] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:53:24.335772 420 external_mini_cluster.cc:1366] Running /tmp/dist-test-task4ezAfy/build/asan/bin/kudu
/tmp/dist-test-task4ezAfy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.62:44705
--webserver_interface=127.0.105.62
--webserver_port=45017
--builtin_ntp_servers=127.0.105.20:38455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729
--hive_metastore_uris=thrift://127.0.0.1:33681
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260501 13:53:24.680987 2564 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260501 13:53:24.681339 2564 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260501 13:53:24.681507 2564 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260501 13:53:24.690423 2564 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260501 13:53:24.690531 2564 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260501 13:53:24.690594 2564 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260501 13:53:24.690713 2564 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260501 13:53:24.701252 2564 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.105.20:38455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:33681
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.62:44705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.105.62
--webserver_port=45017
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision aa7cc0aa9c559b87f264235eaadd03ec492277e4
build type FASTDEBUG
built by None at 01 May 2026 13:43:23 UTC on bdcb31816ec0
build id 11694
ASAN enabled
I20260501 13:53:24.702929 2564 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260501 13:53:24.704828 2564 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260501 13:53:24.713075 2571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260501 13:53:24.713140 2570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260501 13:53:24.713949 2573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260501 13:53:24.714615 2564 server_base.cc:1061] running on GCE node
I20260501 13:53:24.715444 2564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260501 13:53:24.716909 2564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260501 13:53:24.718159 2564 hybrid_clock.cc:648] HybridClock initialized: now 1777643604718094 us; error 54 us; skew 500 ppm
I20260501 13:53:24.718580 2564 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260501 13:53:24.720882 2564 webserver.cc:492] Webserver started at http://127.0.105.62:45017/ using document root <none> and password file <none>
I20260501 13:53:24.721664 2564 fs_manager.cc:362] Metadata directory not provided
I20260501 13:53:24.721803 2564 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260501 13:53:24.727581 2564 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.001s sys 0.002s
I20260501 13:53:24.730628 2579 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260501 13:53:24.732177 2564 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260501 13:53:24.732363 2564 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data,/tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/wal
uuid: "5fa23b4772a944c0b0279eabd7476196"
format_stamp: "Formatted at 2026-05-01 13:52:53 on dist-test-slave-qlwv"
I20260501 13:53:24.733065 2564 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260501 13:53:24.751153 2564 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260501 13:53:24.752290 2564 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260501 13:53:24.752599 2564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260501 13:53:24.773968 2564 rpc_server.cc:307] RPC server started. Bound to: 127.0.105.62:44705
I20260501 13:53:24.774003 2630 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.105.62:44705 every 8 connection(s)
I20260501 13:53:24.775694 2564 server_base.cc:1193] Dumped server information to /tmp/dist-test-task4ezAfy/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777643557648886-420-0/minicluster-data/master-0/data/info.pb
I20260501 13:53:24.776713 420 external_mini_cluster.cc:1428] Started /tmp/dist-test-task4ezAfy/build/asan/bin/kudu as pid 2564
I20260501 13:53:24.784963 2631 sys_catalog.cc:263] Verifying existing consensus state
I20260501 13:53:24.789021 2631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: Bootstrap starting.
I20260501 13:53:24.800829 2631 log.cc:826] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: Log is configured to *not* fsync() on all Append() calls
I20260501 13:53:24.829834 2631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: Bootstrap replayed 1/1 log segments. Stats: ops{read=31 overwritten=0 applied=31 ignored=0} inserts{seen=23 ignored=0} mutations{seen=23 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260501 13:53:24.830533 2631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: Bootstrap complete.
I20260501 13:53:24.836655 2631 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:24.837677 2631 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fa23b4772a944c0b0279eabd7476196, State: Initialized, Role: FOLLOWER
I20260501 13:53:24.838532 2631 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 31, Last appended: 7.31, Last appended by leader: 31, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:24.839457 2635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 7 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } } }
I20260501 13:53:24.839762 2635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: This master's current role is: FOLLOWER
I20260501 13:53:24.840777 2631 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [sys.catalog]: configured and running, proceeding with master startup.
I20260501 13:53:24.854939 2649 catalog_manager.cc:1269] Loaded cluster ID: 6424d308ce1d424b8fe7d2c1b812f4f7
I20260501 13:53:24.855177 2649 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: loading cluster ID for follower catalog manager: success
I20260501 13:53:24.860069 2649 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: acquiring CA information for follower catalog manager: success
I20260501 13:53:24.863492 2649 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260501 13:53:24.872649 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:53:24.876623 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:53:24.891865 1389 heartbeater.cc:344] Connected to a master server at 127.0.105.62:44705
I20260501 13:53:24.894801 2596 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" instance_seqno: 1777643576999180) as {username='slave'} at 127.0.105.3:54377; Asking this server to re-register.
I20260501 13:53:24.895529 1389 heartbeater.cc:461] Registering TS with master...
I20260501 13:53:24.897148 2596 ts_manager.cc:194] Registered new tserver with Master: b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841)
I20260501 13:53:24.912707 1117 heartbeater.cc:344] Connected to a master server at 127.0.105.62:44705
I20260501 13:53:24.914425 2596 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" instance_seqno: 1777643575975284) as {username='slave'} at 127.0.105.1:47571; Asking this server to re-register.
I20260501 13:53:24.915117 1117 heartbeater.cc:461] Registering TS with master...
I20260501 13:53:24.916698 2596 ts_manager.cc:194] Registered new tserver with Master: e4a8a400d10343718a43d5fcc669dd80 (127.0.105.1:44349)
I20260501 13:53:25.127687 1255 heartbeater.cc:344] Connected to a master server at 127.0.105.62:44705
I20260501 13:53:25.129421 2596 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" instance_seqno: 1777643576483530) as {username='slave'} at 127.0.105.2:44023; Asking this server to re-register.
I20260501 13:53:25.130149 1255 heartbeater.cc:461] Registering TS with master...
I20260501 13:53:25.131978 2596 ts_manager.cc:194] Registered new tserver with Master: 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
I20260501 13:53:25.250876 2635 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:25.251201 2635 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:25.253302 2635 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:53:25.254074 2583 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:53:25.255970 2580 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
W20260501 13:53:25.256201 2583 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
I20260501 13:53:25.256487 2583 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:53:25.256886 2635 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:53:25.863830 2657 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:25.864094 2657 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:25.864924 2657 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:53:25.867296 2580 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
W20260501 13:53:25.867475 2583 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
I20260501 13:53:25.867606 2583 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:53:25.867830 2657 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:53:26.944928 2663 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:26.945147 2663 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:26.946023 2663 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:53:26.948203 2583 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:53:26.948436 2580 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:53:26.948553 2580 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:53:26.948822 2663 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:53:27.326958 420 master-stress-test.cc:577] Killing test thread 1/25
I20260501 13:53:27.677976 2668 raft_consensus.cc:493] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:27.678220 2668 raft_consensus.cc:515] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:53:27.678905 2668 leader_election.cc:290] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:53:27.679941 1007 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
W20260501 13:53:27.680137 1005 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
I20260501 13:53:27.680254 1005 leader_election.cc:304] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:53:27.680507 2668 raft_consensus.cc:2749] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:53:27.777964 2669 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:27.778209 2669 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:27.778925 2669 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:53:27.781685 2583 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:53:27.781883 2580 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:53:27.781986 2580 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:53:27.782287 2669 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:53:28.286942 2673 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:28.287144 2673 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:28.287928 2673 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:53:28.290094 2583 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:53:28.290275 2580 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:53:28.290426 2580 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:53:28.290706 2673 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:53:28.854938 2678 raft_consensus.cc:493] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:28.855155 2678 raft_consensus.cc:515] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:53:28.855973 2678 leader_election.cc:290] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:53:28.857074 1005 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
W20260501 13:53:28.857319 1007 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
I20260501 13:53:28.857467 1007 leader_election.cc:304] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:53:28.857803 2678 raft_consensus.cc:2749] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:53:29.879479 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:53:29.879694 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:53:30.025221 2681 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:30.025478 2681 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:30.026209 2681 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:53:30.028103 2583 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:53:30.028326 2580 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:53:30.028432 2580 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:53:30.028641 2681 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:53:30.684149 2684 raft_consensus.cc:493] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:30.684388 2684 raft_consensus.cc:515] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:53:30.685286 2684 leader_election.cc:290] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:53:30.686452 1005 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
W20260501 13:53:30.686661 1007 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
I20260501 13:53:30.686848 1007 leader_election.cc:304] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:53:30.687111 2684 raft_consensus.cc:2749] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:53:31.046736 2684 raft_consensus.cc:493] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:31.046914 2684 raft_consensus.cc:515] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } }
I20260501 13:53:31.047749 2684 leader_election.cc:290] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:53:31.048594 1005 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
W20260501 13:53:31.048795 1007 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
I20260501 13:53:31.048961 1007 leader_election.cc:304] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:53:31.049216 2684 raft_consensus.cc:2749] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:53:31.820266 2688 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:31.820534 2688 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:31.821413 2688 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:53:31.822122 2583 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111) [suppressed 11 similar messages]
W20260501 13:53:31.823647 2583 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:53:31.823832 2580 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:53:31.823982 2580 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:53:31.824262 2688 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:53:33.533736 2693 raft_consensus.cc:493] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:33.533989 2693 raft_consensus.cc:515] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:53:33.534705 2693 leader_election.cc:290] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:53:33.535878 1007 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
W20260501 13:53:33.536098 1005 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
I20260501 13:53:33.536201 1005 leader_election.cc:304] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:53:33.536451 2693 raft_consensus.cc:2749] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:53:34.882375 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:53:34.882575 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:53:36.008590 2700 raft_consensus.cc:493] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:36.008800 2700 raft_consensus.cc:515] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:53:36.009511 2700 leader_election.cc:290] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:53:36.010603 1005 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
W20260501 13:53:36.010859 1007 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
I20260501 13:53:36.010993 1007 leader_election.cc:304] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:53:36.011315 2700 raft_consensus.cc:2749] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:53:37.063294 2703 raft_consensus.cc:493] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:37.063513 2703 raft_consensus.cc:515] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:53:37.064384 2703 leader_election.cc:290] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:53:37.065399 1007 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
W20260501 13:53:37.065778 1005 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
I20260501 13:53:37.065941 1005 leader_election.cc:304] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:53:37.066278 2703 raft_consensus.cc:2749] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:53:38.793072 2707 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:38.793318 2707 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:38.794154 2707 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:53:38.795123 2583 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260501 13:53:38.796698 2583 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:53:38.796864 2580 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:53:38.796998 2580 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:53:38.797264 2707 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:53:39.885536 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:53:39.885792 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:53:42.980903 2723 raft_consensus.cc:493] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:42.981163 2723 raft_consensus.cc:515] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:53:42.981945 2723 leader_election.cc:290] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:53:42.982839 1005 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
W20260501 13:53:42.983018 1007 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
I20260501 13:53:42.983170 1007 leader_election.cc:304] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:53:42.983457 2723 raft_consensus.cc:2749] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:53:43.586866 2724 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:43.587105 2724 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:43.587812 2724 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:53:43.590173 2580 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
W20260501 13:53:43.590327 2583 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
I20260501 13:53:43.590448 2583 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:53:43.590658 2724 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:53:44.888688 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:53:44.889021 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:53:45.856868 2733 raft_consensus.cc:493] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:45.857046 2733 raft_consensus.cc:515] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } }
I20260501 13:53:45.857873 2733 leader_election.cc:290] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:53:45.858685 1005 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
W20260501 13:53:45.858934 1007 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
I20260501 13:53:45.859043 1007 leader_election.cc:304] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:53:45.859385 2733 raft_consensus.cc:2749] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:53:47.929629 2739 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:47.929898 2739 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:47.930737 2739 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:53:47.932021 2580 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260501 13:53:47.933332 2583 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:53:47.933537 2580 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:53:47.933694 2580 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:53:47.934019 2739 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:53:49.892133 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:53:49.892431 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:53:50.943095 2749 raft_consensus.cc:493] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:50.943403 2749 raft_consensus.cc:515] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:53:50.944155 2749 leader_election.cc:290] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:53:50.945156 1005 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
W20260501 13:53:50.945320 1007 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
I20260501 13:53:50.945511 1007 leader_election.cc:304] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:53:50.945773 2749 raft_consensus.cc:2749] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:53:54.899919 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:53:54.900106 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:53:55.368103 2773 raft_consensus.cc:493] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:55.368294 2773 raft_consensus.cc:515] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:53:55.369168 2773 leader_election.cc:290] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:53:55.370074 1007 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
W20260501 13:53:55.370253 1005 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
I20260501 13:53:55.370404 1005 leader_election.cc:304] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:53:55.370657 2773 raft_consensus.cc:2749] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:53:56.245512 2785 raft_consensus.cc:493] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:56.245697 2785 raft_consensus.cc:515] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } }
I20260501 13:53:56.246456 2785 leader_election.cc:290] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:53:56.247447 1005 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
W20260501 13:53:56.247663 1007 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
I20260501 13:53:56.247767 1007 leader_election.cc:304] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:53:56.248013 2785 raft_consensus.cc:2749] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260501 13:53:57.572741 1116 heartbeater.cc:646] Failed to heartbeat to 127.0.105.60:39729 (94 consecutive failures): Network error: Failed to ping master at 127.0.105.60:39729: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
W20260501 13:53:57.651540 1254 heartbeater.cc:646] Failed to heartbeat to 127.0.105.60:39729 (93 consecutive failures): Network error: Failed to ping master at 127.0.105.60:39729: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
W20260501 13:53:57.868811 1388 heartbeater.cc:646] Failed to heartbeat to 127.0.105.60:39729 (93 consecutive failures): Network error: Failed to ping master at 127.0.105.60:39729: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:53:59.867779 2809 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:53:59.868000 2809 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:53:59.868728 2809 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:53:59.869781 2583 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260501 13:53:59.870927 2580 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
W20260501 13:53:59.871078 2583 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
I20260501 13:53:59.871279 2583 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:53:59.871567 2809 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:53:59.902841 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:53:59.903079 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:54:01.172374 2815 raft_consensus.cc:493] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:54:01.172574 2815 raft_consensus.cc:515] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:54:01.173483 2815 leader_election.cc:290] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:54:01.174481 1007 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
W20260501 13:54:01.174813 1005 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
I20260501 13:54:01.174973 1005 leader_election.cc:304] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:54:01.175485 2815 raft_consensus.cc:2749] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:54:02.261566 2823 raft_consensus.cc:493] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:54:02.261754 2823 raft_consensus.cc:515] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:54:02.262411 2823 leader_election.cc:290] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:54:02.263458 1007 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
W20260501 13:54:02.263685 1005 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
I20260501 13:54:02.263906 1005 leader_election.cc:304] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:54:02.264415 2823 raft_consensus.cc:2749] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:54:03.716751 2835 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:54:03.716966 2835 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:54:03.717648 2835 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:54:03.721438 2583 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:54:03.721731 2580 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:54:03.721916 2580 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:54:03.722229 2835 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:54:04.905925 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:54:04.906121 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:54:05.911482 2849 raft_consensus.cc:493] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:54:05.911674 2849 raft_consensus.cc:515] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:54:05.912366 2849 leader_election.cc:290] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:54:05.913352 1005 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
W20260501 13:54:05.913502 1007 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
I20260501 13:54:05.913645 1007 leader_election.cc:304] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:54:05.913925 2849 raft_consensus.cc:2749] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:54:07.747884 2864 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:54:07.748165 2864 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:54:07.748953 2864 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:54:07.749985 2580 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260501 13:54:07.751485 2580 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
W20260501 13:54:07.751679 2583 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
I20260501 13:54:07.751785 2583 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:54:07.752012 2864 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:54:09.161710 2875 raft_consensus.cc:493] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:54:09.161965 2875 raft_consensus.cc:515] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:54:09.162642 2875 leader_election.cc:290] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:54:09.163734 1007 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
W20260501 13:54:09.163894 1005 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
I20260501 13:54:09.164006 1005 leader_election.cc:304] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:54:09.164292 2875 raft_consensus.cc:2749] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:54:09.909439 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:54:09.909716 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:54:13.349726 2899 raft_consensus.cc:493] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:54:13.350006 2899 raft_consensus.cc:515] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:54:13.350728 2899 leader_election.cc:290] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:54:13.351739 1005 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
W20260501 13:54:13.352002 1007 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
I20260501 13:54:13.352169 1007 leader_election.cc:304] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:54:13.352456 2899 raft_consensus.cc:2749] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:54:14.912936 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:54:14.913131 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:54:16.173487 2919 raft_consensus.cc:493] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:54:16.173734 2919 raft_consensus.cc:515] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } }
I20260501 13:54:16.174726 2919 leader_election.cc:290] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:54:16.175849 1007 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
W20260501 13:54:16.176005 1005 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
I20260501 13:54:16.176158 1005 leader_election.cc:304] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:54:16.176467 2919 raft_consensus.cc:2749] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:54:17.927053 2927 raft_consensus.cc:493] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:54:17.927284 2927 raft_consensus.cc:515] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:54:17.927912 2927 leader_election.cc:290] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:54:17.928790 1005 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
W20260501 13:54:17.928966 1007 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
I20260501 13:54:17.929097 1007 leader_election.cc:304] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:54:17.929368 2927 raft_consensus.cc:2749] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:54:19.428860 2939 raft_consensus.cc:493] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:54:19.429059 2939 raft_consensus.cc:515] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:54:19.430181 2939 leader_election.cc:290] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:54:19.430663 1005 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
W20260501 13:54:19.430878 1007 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
I20260501 13:54:19.431012 1007 leader_election.cc:304] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:54:19.431290 2939 raft_consensus.cc:2749] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:54:19.916129 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:54:19.916330 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:54:21.107757 2952 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:54:21.108070 2952 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:54:21.109133 2952 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:54:21.110036 2583 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260501 13:54:21.111517 2583 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:54:21.111663 2580 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:54:21.111827 2580 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:54:21.112063 2952 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:54:23.557915 2970 raft_consensus.cc:493] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:54:23.558115 2970 raft_consensus.cc:515] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:54:23.558866 2970 leader_election.cc:290] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:54:23.559741 1005 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
W20260501 13:54:23.559917 1007 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
I20260501 13:54:23.560046 1007 leader_election.cc:304] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:54:23.560331 2970 raft_consensus.cc:2749] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:54:24.919502 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:54:24.919703 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:54:26.491814 2985 raft_consensus.cc:493] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:54:26.492014 2985 raft_consensus.cc:515] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:54:26.492657 2985 leader_election.cc:290] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:54:26.493675 1007 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
W20260501 13:54:26.493872 1005 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
I20260501 13:54:26.494050 1005 leader_election.cc:304] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:54:26.494359 2985 raft_consensus.cc:2749] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:54:27.736124 2994 raft_consensus.cc:493] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:54:27.736367 2994 raft_consensus.cc:515] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:54:27.737134 2994 leader_election.cc:290] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:54:27.738083 1005 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
W20260501 13:54:27.738284 1007 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
I20260501 13:54:27.738451 1007 leader_election.cc:304] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:54:27.738765 2994 raft_consensus.cc:2749] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:54:29.922430 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:54:29.922667 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:54:31.445588 3016 raft_consensus.cc:493] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:54:31.445818 3016 raft_consensus.cc:515] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } }
I20260501 13:54:31.446729 3016 leader_election.cc:290] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:54:31.447738 1005 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
W20260501 13:54:31.447885 1007 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
I20260501 13:54:31.448068 1007 leader_election.cc:304] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:54:31.448400 3016 raft_consensus.cc:2749] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:54:33.742934 3030 raft_consensus.cc:493] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:54:33.743126 3030 raft_consensus.cc:515] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:54:33.743899 3030 leader_election.cc:290] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:54:33.744994 1005 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
W20260501 13:54:33.745198 1007 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
I20260501 13:54:33.745375 1007 leader_election.cc:304] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:54:33.745633 3030 raft_consensus.cc:2749] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:54:34.926391 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:54:34.926717 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:54:39.352747 3066 raft_consensus.cc:493] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:54:39.352950 3066 raft_consensus.cc:515] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:54:39.353751 3066 leader_election.cc:290] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:54:39.354624 1005 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
W20260501 13:54:39.354794 1007 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
I20260501 13:54:39.354964 1007 leader_election.cc:304] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:54:39.355275 3066 raft_consensus.cc:2749] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:54:39.929951 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:54:39.930150 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:54:39.991823 3073 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:54:39.992058 3073 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:54:39.993499 3073 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:54:39.994009 2583 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260501 13:54:39.995494 2580 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
W20260501 13:54:39.995702 2583 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
I20260501 13:54:39.995899 2583 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:54:39.996205 3073 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:54:40.709031 3079 raft_consensus.cc:493] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:54:40.709218 3079 raft_consensus.cc:515] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:54:40.709875 3079 leader_election.cc:290] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:54:40.710878 1005 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
W20260501 13:54:40.711059 1007 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
I20260501 13:54:40.711167 1007 leader_election.cc:304] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:54:40.711483 3079 raft_consensus.cc:2749] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:54:44.256037 3114 raft_consensus.cc:493] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:54:44.256253 3114 raft_consensus.cc:515] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:54:44.257056 3114 leader_election.cc:290] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:54:44.257941 1005 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
W20260501 13:54:44.258092 1007 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
I20260501 13:54:44.258273 1007 leader_election.cc:304] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:54:44.258498 3114 raft_consensus.cc:2749] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:54:44.933113 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:54:44.933279 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:54:45.245471 3125 raft_consensus.cc:493] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:54:45.245713 3125 raft_consensus.cc:515] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } }
I20260501 13:54:45.246407 3125 leader_election.cc:290] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:54:45.247383 1005 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
W20260501 13:54:45.247575 1007 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
I20260501 13:54:45.247699 1007 leader_election.cc:304] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:54:45.247941 3125 raft_consensus.cc:2749] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:54:48.737478 3165 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:54:48.737704 3165 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:54:48.738562 3165 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:54:48.739615 2583 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260501 13:54:48.740800 2580 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
W20260501 13:54:48.740999 2583 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
I20260501 13:54:48.741132 2583 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:54:48.741438 3165 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:54:49.591311 3174 raft_consensus.cc:493] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:54:49.591507 3174 raft_consensus.cc:515] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:54:49.592527 3174 leader_election.cc:290] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:54:49.593463 1005 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
W20260501 13:54:49.593680 1007 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
I20260501 13:54:49.593817 1007 leader_election.cc:304] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:54:49.594144 3174 raft_consensus.cc:2749] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:54:49.936136 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:54:49.936357 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:54:54.738274 3222 raft_consensus.cc:493] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:54:54.738543 3222 raft_consensus.cc:515] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } }
I20260501 13:54:54.739341 3222 leader_election.cc:290] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:54:54.740288 1005 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
W20260501 13:54:54.740460 1007 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
I20260501 13:54:54.740628 1007 leader_election.cc:304] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:54:54.740914 3222 raft_consensus.cc:2749] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:54:54.939478 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:54:54.939716 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:54:56.651324 3246 raft_consensus.cc:493] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:54:56.651523 3246 raft_consensus.cc:515] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:54:56.652141 3246 leader_election.cc:290] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:54:56.653210 1007 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
W20260501 13:54:56.653383 1005 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
I20260501 13:54:56.653492 1005 leader_election.cc:304] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:54:56.653815 3246 raft_consensus.cc:2749] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260501 13:54:57.577395 1118 heartbeater.cc:646] Failed to heartbeat to 127.0.105.61:33209 (248 consecutive failures): Network error: Failed to ping master at 127.0.105.61:33209: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:54:57.849036 1254 heartbeater.cc:646] Failed to heartbeat to 127.0.105.60:39729 (213 consecutive failures): Network error: Failed to ping master at 127.0.105.60:39729: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
W20260501 13:54:58.043365 1388 heartbeater.cc:646] Failed to heartbeat to 127.0.105.60:39729 (213 consecutive failures): Network error: Failed to ping master at 127.0.105.60:39729: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:54:59.031415 3268 raft_consensus.cc:493] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:54:59.031627 3268 raft_consensus.cc:515] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:54:59.032410 3268 leader_election.cc:290] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:54:59.033382 1007 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
W20260501 13:54:59.033573 1005 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
I20260501 13:54:59.033746 1005 leader_election.cc:304] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:54:59.034006 3268 raft_consensus.cc:2749] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:54:59.943346 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:54:59.943532 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:55:02.162477 3294 raft_consensus.cc:493] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:55:02.162679 3294 raft_consensus.cc:515] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:55:02.163616 3294 leader_election.cc:290] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:55:02.164592 1007 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
W20260501 13:55:02.164814 1005 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
I20260501 13:55:02.165000 1005 leader_election.cc:304] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:55:02.165279 3294 raft_consensus.cc:2749] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:55:04.946776 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:55:04.946957 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:55:05.055584 3334 raft_consensus.cc:493] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:55:05.055763 3334 raft_consensus.cc:515] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } }
I20260501 13:55:05.056654 3334 leader_election.cc:290] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:55:05.057680 1007 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
W20260501 13:55:05.057819 1005 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
I20260501 13:55:05.057961 1005 leader_election.cc:304] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:55:05.058259 3334 raft_consensus.cc:2749] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:55:06.977911 3354 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:55:06.978274 3354 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:55:06.979455 3354 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:55:06.980955 2583 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260501 13:55:06.982331 2583 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:55:06.982493 2580 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:55:06.982649 2580 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:55:06.982903 3354 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:55:09.319293 3386 raft_consensus.cc:493] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:55:09.319510 3386 raft_consensus.cc:515] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:55:09.320255 3386 leader_election.cc:290] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:55:09.321270 1005 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
W20260501 13:55:09.321451 1007 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
I20260501 13:55:09.321622 1007 leader_election.cc:304] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:55:09.321911 3386 raft_consensus.cc:2749] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:55:09.950158 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:55:09.950418 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:55:11.412808 3416 raft_consensus.cc:493] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:55:11.413043 3416 raft_consensus.cc:515] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:55:11.413748 3416 leader_election.cc:290] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:55:11.414939 1005 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
W20260501 13:55:11.415279 1007 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
I20260501 13:55:11.415414 1007 leader_election.cc:304] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:55:11.415786 3416 raft_consensus.cc:2749] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:55:12.616217 3430 raft_consensus.cc:493] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:55:12.616462 3430 raft_consensus.cc:515] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } }
I20260501 13:55:12.617223 3430 leader_election.cc:290] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:55:12.618062 1005 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
W20260501 13:55:12.618247 1007 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
I20260501 13:55:12.618367 1007 leader_election.cc:304] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:55:12.618600 3430 raft_consensus.cc:2749] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:55:14.953787 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:55:14.954010 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:55:18.733603 3521 raft_consensus.cc:493] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:55:18.733824 3521 raft_consensus.cc:515] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:55:18.734436 3521 leader_election.cc:290] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:55:18.735508 1007 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
W20260501 13:55:18.736176 1005 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
I20260501 13:55:18.736280 1005 leader_election.cc:304] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:55:18.736553 3521 raft_consensus.cc:2749] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:55:19.958088 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:55:19.958484 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:55:19.982283 3540 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:55:19.982486 3540 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:55:19.983292 3540 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:55:19.983855 2583 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260501 13:55:19.985037 2583 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:55:19.985271 2580 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:55:19.985417 2580 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:55:19.985687 3540 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:55:21.894420 3568 raft_consensus.cc:493] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:55:21.894660 3568 raft_consensus.cc:515] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:55:21.895711 3568 leader_election.cc:290] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:55:21.896689 1007 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
W20260501 13:55:21.896936 1005 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
I20260501 13:55:21.897053 1005 leader_election.cc:304] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:55:21.897317 3568 raft_consensus.cc:2749] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:55:22.730158 3581 raft_consensus.cc:493] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:55:22.730432 3581 raft_consensus.cc:515] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:55:22.731128 3581 leader_election.cc:290] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:55:22.732136 1007 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
W20260501 13:55:22.732337 1005 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
I20260501 13:55:22.732438 1005 leader_election.cc:304] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:55:22.732712 3581 raft_consensus.cc:2749] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:55:24.933081 2585 maintenance_manager.cc:419] P 5fa23b4772a944c0b0279eabd7476196: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033359
I20260501 13:55:24.968458 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:55:24.969038 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:55:24.989190 2584 maintenance_manager.cc:643] P 5fa23b4772a944c0b0279eabd7476196: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.055s user 0.029s sys 0.000s Metrics: {"bytes_written":12121,"cfile_init":1,"dirs.queue_time_us":403,"dirs.run_cpu_time_us":182,"dirs.run_wall_time_us":811,"drs_written":1,"lbm_read_time_us":68,"lbm_reads_lt_1ms":4,"lbm_write_time_us":700,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":23,"thread_start_us":563,"threads_started":2,"wal-append.queue_time_us":412}
I20260501 13:55:24.990376 2585 maintenance_manager.cc:419] P 5fa23b4772a944c0b0279eabd7476196: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1199 bytes on disk
I20260501 13:55:24.991205 2584 maintenance_manager.cc:643] P 5fa23b4772a944c0b0279eabd7476196: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":90,"lbm_reads_lt_1ms":4}
I20260501 13:55:29.792603 3675 raft_consensus.cc:493] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:55:29.792815 3675 raft_consensus.cc:515] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } }
I20260501 13:55:29.794030 3675 leader_election.cc:290] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:55:29.794464 1005 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
W20260501 13:55:29.794675 1007 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
I20260501 13:55:29.794806 1007 leader_election.cc:304] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:55:29.795061 3675 raft_consensus.cc:2749] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:55:29.971819 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:55:29.972106 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:55:30.392707 3675 raft_consensus.cc:493] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:55:30.392889 3675 raft_consensus.cc:515] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:55:30.393648 3675 leader_election.cc:290] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:55:30.394523 1005 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
W20260501 13:55:30.394739 1007 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
I20260501 13:55:30.394883 1007 leader_election.cc:304] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:55:30.395198 3675 raft_consensus.cc:2749] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:55:34.245541 3732 raft_consensus.cc:493] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:55:34.245733 3732 raft_consensus.cc:515] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:55:34.246344 3732 leader_election.cc:290] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:55:34.247392 1005 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
W20260501 13:55:34.247588 1007 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
I20260501 13:55:34.247677 1007 leader_election.cc:304] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:55:34.247951 3732 raft_consensus.cc:2749] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:55:34.791244 3747 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:55:34.791496 3747 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:55:34.792347 3747 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:55:34.793512 2583 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260501 13:55:34.794946 2583 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:55:34.795109 2580 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:55:34.795341 2580 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:55:34.796006 3747 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:55:34.974655 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:55:34.974824 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:55:37.996222 3794 raft_consensus.cc:493] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:55:37.996428 3794 raft_consensus.cc:515] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:55:37.997275 3794 leader_election.cc:290] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:55:37.998162 1005 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
W20260501 13:55:37.998425 1007 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
I20260501 13:55:37.998571 1007 leader_election.cc:304] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:55:37.998880 3794 raft_consensus.cc:2749] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:55:38.098584 3797 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:55:38.098781 3797 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:55:38.099489 3797 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:55:38.101701 2580 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
W20260501 13:55:38.101917 2583 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
I20260501 13:55:38.102068 2583 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:55:38.102293 3797 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:55:39.978389 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:55:39.978570 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:55:43.640281 3855 raft_consensus.cc:493] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:55:43.640518 3855 raft_consensus.cc:515] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:55:43.641261 3855 leader_election.cc:290] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:55:43.642354 1007 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
W20260501 13:55:43.642596 1005 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
I20260501 13:55:43.642699 1005 leader_election.cc:304] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:55:43.642938 3855 raft_consensus.cc:2749] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:55:44.091841 3855 raft_consensus.cc:493] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:55:44.092087 3855 raft_consensus.cc:515] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } }
I20260501 13:55:44.092821 3855 leader_election.cc:290] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:55:44.093700 1005 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
W20260501 13:55:44.093902 1007 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
I20260501 13:55:44.094058 1007 leader_election.cc:304] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:55:44.094331 3855 raft_consensus.cc:2749] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:55:44.981745 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:55:44.981937 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:55:45.225394 3866 raft_consensus.cc:493] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:55:45.225628 3866 raft_consensus.cc:515] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:55:45.226325 3866 leader_election.cc:290] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:55:45.227314 1007 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
W20260501 13:55:45.227504 1005 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
I20260501 13:55:45.227625 1005 leader_election.cc:304] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:55:45.227849 3866 raft_consensus.cc:2749] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:55:48.175201 3909 raft_consensus.cc:493] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:55:48.175503 3909 raft_consensus.cc:515] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:55:48.176223 3909 leader_election.cc:290] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:55:48.177145 1007 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
W20260501 13:55:48.177323 1005 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
I20260501 13:55:48.177498 1005 leader_election.cc:304] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:55:48.177773 3909 raft_consensus.cc:2749] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:55:49.307505 3920 raft_consensus.cc:493] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:55:49.307780 3920 raft_consensus.cc:515] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:55:49.308482 3920 leader_election.cc:290] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:55:49.309448 1005 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
W20260501 13:55:49.309690 1007 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
I20260501 13:55:49.309834 1007 leader_election.cc:304] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:55:49.310170 3920 raft_consensus.cc:2749] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:55:49.501212 3920 raft_consensus.cc:493] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:55:49.501475 3920 raft_consensus.cc:515] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } }
I20260501 13:55:49.502146 3920 leader_election.cc:290] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:55:49.503139 1007 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
W20260501 13:55:49.503343 1005 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
I20260501 13:55:49.503509 1005 leader_election.cc:304] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:55:49.503829 3920 raft_consensus.cc:2749] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:55:49.605643 3930 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:55:49.605919 3930 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:55:49.606739 3930 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:55:49.608057 2583 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:55:49.608086 2580 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260501 13:55:49.609579 2580 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
W20260501 13:55:49.609760 2583 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
I20260501 13:55:49.609892 2583 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:55:49.610124 3930 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:55:49.984781 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:55:49.984954 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:55:50.286090 3943 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:55:50.286360 3943 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:55:50.287078 3943 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:55:50.288933 2583 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:55:50.289217 2580 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:55:50.289363 2580 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:55:50.289764 3943 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:55:53.380915 3976 raft_consensus.cc:493] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:55:53.381157 3976 raft_consensus.cc:515] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:55:53.382109 3976 leader_election.cc:290] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:55:53.383132 1007 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
W20260501 13:55:53.383315 1005 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
I20260501 13:55:53.383452 1005 leader_election.cc:304] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:55:53.383708 3976 raft_consensus.cc:2749] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:55:54.939284 3994 raft_consensus.cc:493] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:55:54.939577 3994 raft_consensus.cc:515] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:55:54.940357 3994 leader_election.cc:290] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:55:54.941368 1007 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
W20260501 13:55:54.941711 1005 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
I20260501 13:55:54.941841 1005 leader_election.cc:304] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:55:54.942096 3994 raft_consensus.cc:2749] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:55:54.987761 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:55:54.987980 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:55:56.964497 4016 raft_consensus.cc:493] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:55:56.964740 4016 raft_consensus.cc:515] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:55:56.965525 4016 leader_election.cc:290] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:55:56.966431 1007 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
W20260501 13:55:56.966672 1005 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
I20260501 13:55:56.966792 1005 leader_election.cc:304] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:55:56.967056 4016 raft_consensus.cc:2749] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260501 13:55:57.727069 1118 heartbeater.cc:646] Failed to heartbeat to 127.0.105.61:33209 (368 consecutive failures): Network error: Failed to ping master at 127.0.105.61:33209: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:55:58.040251 1256 heartbeater.cc:646] Failed to heartbeat to 127.0.105.61:33209 (361 consecutive failures): Network error: Failed to ping master at 127.0.105.61:33209: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:55:58.230778 1388 heartbeater.cc:646] Failed to heartbeat to 127.0.105.60:39729 (333 consecutive failures): Network error: Failed to ping master at 127.0.105.60:39729: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:55:59.991011 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:55:59.991230 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:56:03.689925 4104 raft_consensus.cc:493] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:56:03.690201 4104 raft_consensus.cc:515] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } }
I20260501 13:56:03.690898 4104 leader_election.cc:290] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:56:03.691982 1007 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
W20260501 13:56:03.692210 1005 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
I20260501 13:56:03.692354 1005 leader_election.cc:304] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:56:03.692626 4104 raft_consensus.cc:2749] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:56:04.994282 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:56:04.994553 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:56:06.599263 4142 raft_consensus.cc:493] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:56:06.599479 4142 raft_consensus.cc:515] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:56:06.600328 4142 leader_election.cc:290] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:56:06.601269 1005 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
W20260501 13:56:06.601502 1007 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
I20260501 13:56:06.601630 1007 leader_election.cc:304] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:56:06.601919 4142 raft_consensus.cc:2749] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:56:06.605211 4145 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:56:06.605427 4145 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:56:06.606448 4145 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:56:06.607540 2583 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111) [suppressed 2 similar messages]
W20260501 13:56:06.608953 2583 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:56:06.609165 2580 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:56:06.609288 2580 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:56:06.609597 4145 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:56:07.354723 4159 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:56:07.354997 4159 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:56:07.355865 4159 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:56:07.357810 2583 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:56:07.358060 2580 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:56:07.358214 2580 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:56:07.358479 4159 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:56:09.997680 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:56:09.997905 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:56:10.253752 4201 raft_consensus.cc:493] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:56:10.253937 4201 raft_consensus.cc:515] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:56:10.255398 4201 leader_election.cc:290] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:56:10.255812 1007 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
W20260501 13:56:10.256098 1005 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
I20260501 13:56:10.256201 1005 leader_election.cc:304] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:56:10.256537 4201 raft_consensus.cc:2749] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:56:13.894569 4244 raft_consensus.cc:493] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:56:13.894814 4244 raft_consensus.cc:515] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } }
I20260501 13:56:13.895576 4244 leader_election.cc:290] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:56:13.896585 1005 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
W20260501 13:56:13.896756 1007 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
I20260501 13:56:13.896865 1007 leader_election.cc:304] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:56:13.897111 4244 raft_consensus.cc:2749] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:56:14.018354 4251 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:56:14.018610 4251 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:56:14.019433 4251 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:56:14.021291 2583 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260501 13:56:14.022768 2583 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:56:14.023002 2580 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:56:14.023123 2580 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:56:14.023454 4251 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:56:15.000855 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:56:15.001037 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:56:15.614390 4267 raft_consensus.cc:493] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:56:15.614662 4267 raft_consensus.cc:515] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:56:15.615607 4267 leader_election.cc:290] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:56:15.616734 1007 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
W20260501 13:56:15.616992 1005 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
I20260501 13:56:15.617164 1005 leader_election.cc:304] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:56:15.617496 4267 raft_consensus.cc:2749] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:56:16.393170 4267 raft_consensus.cc:493] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:56:16.393379 4267 raft_consensus.cc:515] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:56:16.394110 4267 leader_election.cc:290] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:56:16.395069 1005 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
W20260501 13:56:16.395287 1007 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
I20260501 13:56:16.395435 1007 leader_election.cc:304] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:56:16.395713 4267 raft_consensus.cc:2749] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:56:19.000505 4317 raft_consensus.cc:493] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:56:19.000763 4317 raft_consensus.cc:515] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:56:19.001557 4317 leader_election.cc:290] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:56:19.002671 1007 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
W20260501 13:56:19.002887 1005 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
I20260501 13:56:19.003078 1005 leader_election.cc:304] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:56:19.003487 4317 raft_consensus.cc:2749] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:56:20.004447 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:56:20.004688 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:56:23.239001 4379 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:56:23.239293 4379 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:56:23.240054 4379 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:56:23.241196 2583 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260501 13:56:23.242514 2583 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:56:23.242712 2580 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:56:23.242854 2580 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:56:23.243155 4379 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:56:24.669948 4392 raft_consensus.cc:493] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:56:24.670125 4392 raft_consensus.cc:515] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:56:24.670850 4392 leader_election.cc:290] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:56:24.671875 1007 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
W20260501 13:56:24.672063 1005 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
I20260501 13:56:24.672191 1005 leader_election.cc:304] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:56:24.672451 4392 raft_consensus.cc:2749] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:56:24.867630 4392 raft_consensus.cc:493] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:56:24.867868 4392 raft_consensus.cc:515] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } }
I20260501 13:56:24.868572 4392 leader_election.cc:290] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:56:24.869426 1005 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
W20260501 13:56:24.869604 1007 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
I20260501 13:56:24.869724 1007 leader_election.cc:304] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:56:24.870656 4392 raft_consensus.cc:2749] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:56:25.007781 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:56:25.007999 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:56:27.031752 4432 raft_consensus.cc:493] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:56:27.031980 4432 raft_consensus.cc:515] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:56:27.032912 4432 leader_election.cc:290] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:56:27.033771 1005 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
W20260501 13:56:27.033988 1007 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
I20260501 13:56:27.034114 1007 leader_election.cc:304] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:56:27.034389 4432 raft_consensus.cc:2749] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:56:28.039000 4443 raft_consensus.cc:493] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:56:28.039276 4443 raft_consensus.cc:515] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:56:28.040772 4443 leader_election.cc:290] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:56:28.041016 1007 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
W20260501 13:56:28.041195 1005 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
I20260501 13:56:28.041339 1005 leader_election.cc:304] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:56:28.041644 4443 raft_consensus.cc:2749] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:56:29.255810 4455 raft_consensus.cc:493] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:56:29.256006 4455 raft_consensus.cc:515] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:56:29.256709 4455 leader_election.cc:290] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:56:29.257720 1005 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
W20260501 13:56:29.257865 1007 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
I20260501 13:56:29.258056 1007 leader_election.cc:304] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:56:29.258308 4455 raft_consensus.cc:2749] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:56:29.320145 4461 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:56:29.320393 4461 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:56:29.321203 4461 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:56:29.322188 2583 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260501 13:56:29.323449 2580 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
W20260501 13:56:29.323650 2583 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
I20260501 13:56:29.323763 2583 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:56:29.324020 4461 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:56:30.011315 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:56:30.011488 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:56:32.420948 4508 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:56:32.421139 4508 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:56:32.422343 4508 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:56:32.424695 2583 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:56:32.424939 2580 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:56:32.425073 2580 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:56:32.425405 4508 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:56:35.014649 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:56:35.014896 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:56:36.378154 4553 raft_consensus.cc:493] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:56:36.378420 4553 raft_consensus.cc:515] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:56:36.379211 4553 leader_election.cc:290] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:56:36.380276 1005 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
W20260501 13:56:36.380960 1007 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
I20260501 13:56:36.381107 1007 leader_election.cc:304] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:56:36.381428 4553 raft_consensus.cc:2749] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:56:39.867352 4605 raft_consensus.cc:493] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:56:39.867560 4605 raft_consensus.cc:515] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } }
I20260501 13:56:39.868294 4605 leader_election.cc:290] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:56:39.869086 1005 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
W20260501 13:56:39.869273 1007 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
I20260501 13:56:39.869436 1007 leader_election.cc:304] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:56:39.869699 4605 raft_consensus.cc:2749] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:56:40.018054 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:56:40.018304 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:56:42.129261 4637 raft_consensus.cc:493] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:56:42.129460 4637 raft_consensus.cc:515] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:56:42.130268 4637 leader_election.cc:290] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:56:42.131439 1007 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
W20260501 13:56:42.131690 1005 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
I20260501 13:56:42.131812 1005 leader_election.cc:304] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:56:42.132182 4637 raft_consensus.cc:2749] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:56:45.021361 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:56:45.021596 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:56:45.773200 4678 raft_consensus.cc:493] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:56:45.773392 4678 raft_consensus.cc:515] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:56:45.774142 4678 leader_election.cc:290] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:56:45.775331 1007 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
W20260501 13:56:45.775671 1005 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
I20260501 13:56:45.775785 1005 leader_election.cc:304] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:56:45.776114 4678 raft_consensus.cc:2749] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:56:47.431005 4704 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:56:47.431288 4704 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:56:47.432193 4704 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:56:47.433507 2580 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260501 13:56:47.435268 2583 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:56:47.435490 2580 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:56:47.435617 2580 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:56:47.435887 4704 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:56:50.024569 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:56:50.024773 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:56:50.613250 4734 raft_consensus.cc:493] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:56:50.613464 4734 raft_consensus.cc:515] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:56:50.614185 4734 leader_election.cc:290] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:56:50.615144 1005 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
W20260501 13:56:50.615482 1007 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
I20260501 13:56:50.615655 1007 leader_election.cc:304] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:56:50.615955 4734 raft_consensus.cc:2749] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:56:55.027752 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:56:55.027961 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:56:55.692070 4801 raft_consensus.cc:493] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:56:55.692284 4801 raft_consensus.cc:515] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:56:55.692996 4801 leader_election.cc:290] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:56:55.694023 1005 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
W20260501 13:56:55.694180 1007 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
I20260501 13:56:55.694365 1007 leader_election.cc:304] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:56:55.694687 4801 raft_consensus.cc:2749] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:56:57.203574 4815 raft_consensus.cc:493] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:56:57.203759 4815 raft_consensus.cc:515] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } }
I20260501 13:56:57.204631 4815 leader_election.cc:290] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:56:57.205678 1007 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
W20260501 13:56:57.205936 1005 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
I20260501 13:56:57.206102 1005 leader_election.cc:304] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:56:57.206418 4815 raft_consensus.cc:2749] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260501 13:56:57.890465 1118 heartbeater.cc:646] Failed to heartbeat to 127.0.105.61:33209 (488 consecutive failures): Network error: Failed to ping master at 127.0.105.61:33209: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:56:58.258525 1256 heartbeater.cc:646] Failed to heartbeat to 127.0.105.61:33209 (481 consecutive failures): Network error: Failed to ping master at 127.0.105.61:33209: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:56:58.427160 1388 heartbeater.cc:646] Failed to heartbeat to 127.0.105.60:39729 (453 consecutive failures): Network error: Failed to ping master at 127.0.105.60:39729: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:56:58.719537 4841 raft_consensus.cc:493] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:56:58.719739 4841 raft_consensus.cc:515] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:56:58.720503 4841 leader_election.cc:290] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:56:58.721480 1005 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
W20260501 13:56:58.721701 1007 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
I20260501 13:56:58.721817 1007 leader_election.cc:304] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:56:58.722146 4841 raft_consensus.cc:2749] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:56:59.149369 4841 raft_consensus.cc:493] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:56:59.149567 4841 raft_consensus.cc:515] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:56:59.150311 4841 leader_election.cc:290] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:56:59.151175 1005 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
W20260501 13:56:59.151358 1007 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
I20260501 13:56:59.151503 1007 leader_election.cc:304] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:56:59.151803 4841 raft_consensus.cc:2749] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:56:59.726668 4853 raft_consensus.cc:493] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:56:59.727022 4853 raft_consensus.cc:515] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:56:59.727764 4853 leader_election.cc:290] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:56:59.728885 1007 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
W20260501 13:56:59.729084 1005 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
I20260501 13:56:59.729182 1005 leader_election.cc:304] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:56:59.729408 4853 raft_consensus.cc:2749] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:57:00.030979 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:57:00.031198 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:57:01.016592 4873 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:57:01.016829 4873 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:57:01.017581 4873 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:57:01.019248 2583 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260501 13:57:01.020588 2580 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
W20260501 13:57:01.020764 2583 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
I20260501 13:57:01.020915 2583 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:57:01.021200 4873 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:57:04.157843 4917 raft_consensus.cc:493] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:57:04.158035 4917 raft_consensus.cc:515] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:57:04.158860 4917 leader_election.cc:290] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:57:04.159952 1005 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
W20260501 13:57:04.160161 1007 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
I20260501 13:57:04.160302 1007 leader_election.cc:304] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:57:04.160547 4917 raft_consensus.cc:2749] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:57:05.034128 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:57:05.034340 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:57:07.760001 4963 raft_consensus.cc:493] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:57:07.760195 4963 raft_consensus.cc:515] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:57:07.761015 4963 leader_election.cc:290] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:57:07.762035 1005 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
W20260501 13:57:07.762233 1007 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
I20260501 13:57:07.762424 1007 leader_election.cc:304] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:57:07.762717 4963 raft_consensus.cc:2749] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:57:08.654326 4975 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:57:08.654548 4975 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:57:08.655313 4975 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:57:08.656361 2583 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260501 13:57:08.657526 2580 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
W20260501 13:57:08.657720 2583 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
I20260501 13:57:08.657860 2583 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:57:08.658150 4975 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:57:10.037349 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:57:10.037604 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:57:10.693128 5000 raft_consensus.cc:493] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:57:10.693365 5000 raft_consensus.cc:515] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } }
I20260501 13:57:10.694236 5000 leader_election.cc:290] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:57:10.695204 1005 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
W20260501 13:57:10.695453 1007 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
I20260501 13:57:10.695608 1007 leader_election.cc:304] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:57:10.695890 5000 raft_consensus.cc:2749] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:57:12.580538 5030 raft_consensus.cc:493] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:57:12.580771 5030 raft_consensus.cc:515] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:57:12.581684 5030 leader_election.cc:290] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:57:12.582623 1005 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
W20260501 13:57:12.582835 1007 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
I20260501 13:57:12.582999 1007 leader_election.cc:304] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:57:12.583343 5030 raft_consensus.cc:2749] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:57:12.800225 5030 raft_consensus.cc:493] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:57:12.800459 5030 raft_consensus.cc:515] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:57:12.801167 5030 leader_election.cc:290] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:57:12.802143 1005 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
W20260501 13:57:12.802314 1007 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
I20260501 13:57:12.802477 1007 leader_election.cc:304] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:57:12.802809 5030 raft_consensus.cc:2749] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:57:15.040984 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:57:15.041254 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:57:15.994396 5074 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:57:15.994592 5074 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:57:15.995411 5074 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:57:15.996430 2583 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260501 13:57:15.997795 2580 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
W20260501 13:57:15.998006 2583 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
I20260501 13:57:15.998162 2583 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:57:15.998427 5074 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:57:18.609419 5112 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:57:18.609601 5112 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:57:18.610165 5112 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:57:18.612495 2583 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:57:18.612658 2580 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:57:18.612828 2580 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:57:18.613117 5112 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:57:19.154516 5111 raft_consensus.cc:493] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:57:19.154744 5111 raft_consensus.cc:515] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:57:19.155529 5111 leader_election.cc:290] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:57:19.156433 1005 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
W20260501 13:57:19.156631 1007 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
I20260501 13:57:19.156796 1007 leader_election.cc:304] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:57:19.157105 5111 raft_consensus.cc:2749] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:57:20.044854 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:57:20.045110 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:57:23.566854 5170 raft_consensus.cc:493] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:57:23.567101 5170 raft_consensus.cc:515] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:57:23.567809 5170 leader_election.cc:290] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:57:23.568693 1007 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
W20260501 13:57:23.568895 1005 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
I20260501 13:57:23.569000 1005 leader_election.cc:304] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:57:23.569216 5170 raft_consensus.cc:2749] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:57:25.048080 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:57:25.048273 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:57:26.225323 5215 raft_consensus.cc:493] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:57:26.225571 5215 raft_consensus.cc:515] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } }
I20260501 13:57:26.226363 5215 leader_election.cc:290] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:57:26.227268 1007 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
W20260501 13:57:26.227440 1005 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
I20260501 13:57:26.227562 1005 leader_election.cc:304] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:57:26.227815 5215 raft_consensus.cc:2749] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:57:26.281582 5216 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:57:26.281870 5216 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:57:26.282665 5216 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:57:26.284238 2583 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260501 13:57:26.285501 2583 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:57:26.285681 2580 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:57:26.285797 2580 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:57:26.286041 5216 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:57:30.051288 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:57:30.051481 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:57:31.454519 5280 raft_consensus.cc:493] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:57:31.454746 5280 raft_consensus.cc:515] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:57:31.455664 5280 leader_election.cc:290] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:57:31.456447 1005 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
W20260501 13:57:31.456609 1007 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
I20260501 13:57:31.456779 1007 leader_election.cc:304] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:57:31.457037 5280 raft_consensus.cc:2749] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:57:34.965811 5323 raft_consensus.cc:493] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:57:34.966046 5323 raft_consensus.cc:515] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:57:34.966817 5323 leader_election.cc:290] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:57:34.967854 1007 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
W20260501 13:57:34.968065 1005 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
I20260501 13:57:34.968211 1005 leader_election.cc:304] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:57:34.968564 5323 raft_consensus.cc:2749] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:57:35.055303 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:57:35.055541 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:57:36.424592 5350 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:57:36.424819 5350 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:57:36.425673 5350 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:57:36.426759 2583 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260501 13:57:36.428319 2583 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:57:36.428625 2580 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:57:36.428790 2580 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:57:36.429124 5350 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:57:36.838863 5354 raft_consensus.cc:493] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:57:36.839159 5354 raft_consensus.cc:515] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } }
I20260501 13:57:36.839891 5354 leader_election.cc:290] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:57:36.840955 1007 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
W20260501 13:57:36.841156 1005 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
I20260501 13:57:36.841275 1005 leader_election.cc:304] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:57:36.841606 5354 raft_consensus.cc:2749] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:57:37.942786 5354 raft_consensus.cc:493] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:57:37.943049 5354 raft_consensus.cc:515] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:57:37.943795 5354 leader_election.cc:290] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:57:37.944962 1005 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
W20260501 13:57:37.945183 1007 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
I20260501 13:57:37.945344 1007 leader_election.cc:304] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:57:37.945782 5354 raft_consensus.cc:2749] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:57:40.058561 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:57:40.058771 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:57:41.135351 5413 raft_consensus.cc:493] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:57:41.135564 5413 raft_consensus.cc:515] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:57:41.136665 5413 leader_election.cc:290] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:57:41.137686 1005 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
W20260501 13:57:41.137856 1007 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
I20260501 13:57:41.138037 1007 leader_election.cc:304] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:57:41.138310 5413 raft_consensus.cc:2749] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:57:45.062282 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:57:45.062527 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:57:46.753460 5486 raft_consensus.cc:493] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:57:46.753712 5486 raft_consensus.cc:515] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:57:46.754609 5486 leader_election.cc:290] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:57:46.755748 1007 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
W20260501 13:57:46.755905 1005 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
I20260501 13:57:46.756086 1005 leader_election.cc:304] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:57:46.756386 5486 raft_consensus.cc:2749] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:57:50.065279 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:57:50.065449 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:57:51.302368 5546 raft_consensus.cc:493] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:57:51.302578 5546 raft_consensus.cc:515] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:57:51.303804 5546 leader_election.cc:290] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:57:51.304899 1007 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
W20260501 13:57:51.305088 1005 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
I20260501 13:57:51.305254 1005 leader_election.cc:304] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:57:51.305512 5546 raft_consensus.cc:2749] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:57:51.914223 5554 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:57:51.914440 5554 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa23b4772a944c0b0279eabd7476196" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 44705 } } peers { permanent_uuid: "84a39a5f167243ff9bfe07189aa881ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33209 } } peers { permanent_uuid: "e47102ddbd4f41c1bf0a2864d97237b7" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 39729 } }
I20260501 13:57:51.915292 5554 leader_election.cc:290] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209), e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729)
W20260501 13:57:51.916621 2583 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260501 13:57:51.917984 2583 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84a39a5f167243ff9bfe07189aa881ca (127.0.105.61:33209): Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:57:51.918220 2580 leader_election.cc:336] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e47102ddbd4f41c1bf0a2864d97237b7 (127.0.105.60:39729): Network error: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:57:51.918350 2580 leader_election.cc:304] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fa23b4772a944c0b0279eabd7476196; no voters: 84a39a5f167243ff9bfe07189aa881ca, e47102ddbd4f41c1bf0a2864d97237b7
I20260501 13:57:51.918579 5554 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5fa23b4772a944c0b0279eabd7476196 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260501 13:57:52.091394 5562 raft_consensus.cc:493] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:57:52.091614 5562 raft_consensus.cc:515] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } } peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } }
I20260501 13:57:52.092211 5562 leader_election.cc:290] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:57:52.093088 1005 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
W20260501 13:57:52.093289 1007 leader_election.cc:343] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: b1231946d86d49caa3c9fcb632b38896
I20260501 13:57:52.093470 1007 leader_election.cc:304] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:57:52.093763 5562 raft_consensus.cc:2749] T b1231946d86d49caa3c9fcb632b38896 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:57:55.068542 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:57:55.068768 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260501 13:57:58.046622 1118 heartbeater.cc:646] Failed to heartbeat to 127.0.105.61:33209 (608 consecutive failures): Network error: Failed to ping master at 127.0.105.61:33209: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:57:58.477898 1256 heartbeater.cc:646] Failed to heartbeat to 127.0.105.61:33209 (601 consecutive failures): Network error: Failed to ping master at 127.0.105.61:33209: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111)
W20260501 13:57:58.627677 1388 heartbeater.cc:646] Failed to heartbeat to 127.0.105.60:39729 (573 consecutive failures): Network error: Failed to ping master at 127.0.105.60:39729: Client connection negotiation failed: client connection to 127.0.105.60:39729: connect: Connection refused (error 111)
I20260501 13:57:58.789366 5640 raft_consensus.cc:493] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:57:58.789562 5640 raft_consensus.cc:515] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:57:58.790230 5640 leader_election.cc:290] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:57:58.791239 1007 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
W20260501 13:57:58.791427 1005 leader_election.cc:343] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: 31bef174b83549ad95990b0ddb64360f
I20260501 13:57:58.791543 1005 leader_election.cc:304] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:57:58.791853 5640 raft_consensus.cc:2749] T 31bef174b83549ad95990b0ddb64360f P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:58:00.071818 2564 master_runner.cc:186] Error getting master registration for 127.0.105.61:33209: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33209: connect: Connection refused (error 111),
I20260501 13:58:00.072042 2564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260501 13:58:00.429934 5665 raft_consensus.cc:493] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:58:00.430160 5665 raft_consensus.cc:515] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:58:00.430946 5665 leader_election.cc:290] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:58:00.431782 1005 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
W20260501 13:58:00.432036 1007 leader_election.cc:343] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: bcf6cd634f444b1d90c0440134510d1e
I20260501 13:58:00.432168 1007 leader_election.cc:304] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:58:00.432483 5665 raft_consensus.cc:2749] T bcf6cd634f444b1d90c0440134510d1e P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260501 13:58:02.475479 5692 raft_consensus.cc:493] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260501 13:58:02.475684 5692 raft_consensus.cc:515] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0226350eae044398f04ef4e5cbb550e" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41841 } } peers { permanent_uuid: "1151db12ecae4dd7bb88b3d1415b7d92" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43565 } } peers { permanent_uuid: "e4a8a400d10343718a43d5fcc669dd80" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 44349 } }
I20260501 13:58:02.476516 5692 leader_election.cc:290] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841), 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565)
W20260501 13:58:02.477596 1007 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1151db12ecae4dd7bb88b3d1415b7d92 (127.0.105.2:43565): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
W20260501 13:58:02.477785 1005 leader_election.cc:343] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0226350eae044398f04ef4e5cbb550e (127.0.105.3:41841): Not found: Tablet not found: cf800b1230eb4b5eac7feb0276d54486
I20260501 13:58:02.477983 1005 leader_election.cc:304] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4a8a400d10343718a43d5fcc669dd80; no voters: 1151db12ecae4dd7bb88b3d1415b7d92, b0226350eae044398f04ef4e5cbb550e
I20260501 13:58:02.478262 5692 raft_consensus.cc:2749] T cf800b1230eb4b5eac7feb0276d54486 P e4a8a400d10343718a43d5fcc669dd80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260501 13:58:02.826737 1398 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)
W20260501 13:58:02.827142 1398 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)
W20260501 13:58:02.827370 1398 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)
W20260501 13:58:02.827605 1398 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)
F20260501 13:58:02.838619 1421 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)
*** Check failure stack trace: ***
*** Aborted at 1777643882 (unix time) try "date -d @1777643882" if you are using GNU date ***
F20260501 13:58:02.840389 1413 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2e04f40a058b4f08a86ed4ca4cd566cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)
*** Check failure stack trace: ***
F20260501 13:58:02.840389 1413 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2e04f40a058b4f08a86ed4ca4cd566cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.840448 1420 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)
*** Check failure stack trace: ***
F20260501 13:58:02.840389 1413 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2e04f40a058b4f08a86ed4ca4cd566cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.840448 1420 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.840445 1418 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)
*** Check failure stack trace: ***
F20260501 13:58:02.840389 1413 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2e04f40a058b4f08a86ed4ca4cd566cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.840448 1420 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.840445 1418 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.841805 1431 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F2*** Check failure stack trace: ***
F20260501 13:58:02.840389 1413 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2e04f40a058b4f08a86ed4ca4cd566cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.840448 1420 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.840445 1418 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.841805 1431 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.841912 1407 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b3375377b39846b986b09a1f8fd0b606 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)
*** Check failure stack trace: ***
F20260501 13:58:02.840389 1413 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2e04f40a058b4f08a86ed4ca4cd566cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.840448 1420 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.840445 1418 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.841805 1431 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.841912 1407 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b3375377b39846b986b09a1f8fd0b606 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842144 1406 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_215a35e1b7404ba1aa928f075959f1e5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1) 20260501 13:58:02.842183 1414 master-stress-test.cc:297] Check failed: _s.ok() Bad status: T*** Check failure stack trace: ***
F20260501 13:58:02.840389 1413 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2e04f40a058b4f08a86ed4ca4cd566cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.840448 1420 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.840445 1418 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.841805 1431 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.841912 1407 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b3375377b39846b986b09a1f8fd0b606 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842144 1406 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_215a35e1b7404ba1aa928f075959f1e5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1) 20260501 13:58:02.842183 1414 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b93ea2b7623547349398aaf4f3d85376 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)
*** Check failure stack trace: ***
F20260501 13:58:02.840389 1413 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2e04f40a058b4f08a86ed4ca4cd566cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.840448 1420 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.840445 1418 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.841805 1431 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.841912 1407 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b3375377b39846b986b09a1f8fd0b606 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842144 1406 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_215a35e1b7404ba1aa928f075959f1e5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1) 20260501 13:58:02.842183 1414 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b93ea2b7623547349398aaf4f3d85376 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842605 1416 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)
*** Check failure stack trace: ***
F20260501 13:58:02.840389 1413 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2e04f40a058b4f08a86ed4ca4cd566cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.840448 1420 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.840445 1418 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.841805 1431 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.841912 1407 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b3375377b39846b986b09a1f8fd0b606 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842144 1406 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_215a35e1b7404ba1aa928f075959f1e5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1) 20260501 13:58:02.842183 1414 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b93ea2b7623547349398aaf4f3d85376 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842605 1416 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842679 1419 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)
*** Check failure stack trace: ***
F20260501 13:58:02.840389 1413 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2e04f40a058b4f08a86ed4ca4cd566cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.840448 1420 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.840445 1418 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.841805 1431 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.841912 1407 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b3375377b39846b986b09a1f8fd0b606 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842144 1406 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_215a35e1b7404ba1aa928f075959f1e5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1) 20260501 13:58:02.842183 1414 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b93ea2b7623547349398aaf4f3d85376 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842605 1416 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842679 1419 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842983 1411 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2ff058e2a2324419b03d708582f033d5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)
*** Check failure stack trace: ***
F20260501 13:58:02.840389 1413 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2e04f40a058b4f08a86ed4ca4cd566cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.840448 1420 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.840445 1418 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.841805 1431 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.841912 1407 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b3375377b39846b986b09a1f8fd0b606 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842144 1406 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_215a35e1b7404ba1aa928f075959f1e5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1) 20260501 13:58:02.842183 1414 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b93ea2b7623547349398aaf4f3d85376 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842605 1416 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842679 1419 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842983 1411 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2ff058e2a2324419b03d708582f033d5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842998 1427 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)
*** Check failure stack trace: ***
PC: @ 0x0 (unknown)
*** SIGABRT (@0x3e8000001a4) received by PID 420 (TID 0x7f50603f0700) from PID 420; stack trace: ***
F20260501 13:58:02.840389 1413 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2e04f40a058b4f08a86ed4ca4cd566cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.840448 1420 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.840445 1418 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.841805 1431 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.841912 1407 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b3375377b39846b986b09a1f8fd0b606 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842144 1406 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_215a35e1b7404ba1aa928f075959f1e5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1) 20260501 13:58:02.842183 1414 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b93ea2b7623547349398aaf4f3d85376 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842605 1416 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842679 1419 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842983 1411 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2ff058e2a2324419b03d708582f033d5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842998 1427 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.846299 1415 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_3e7f39a3248e4963bda4b7790c673d73 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)
*** Check failure stack trace: ***
F20260501 13:58:02.840389 1413 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2e04f40a058b4f08a86ed4ca4cd566cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.840448 1420 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.840445 1418 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.841805 1431 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.841912 1407 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b3375377b39846b986b09a1f8fd0b606 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842144 1406 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_215a35e1b7404ba1aa928f075959f1e5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1) 20260501 13:58:02.842183 1414 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b93ea2b7623547349398aaf4f3d85376 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842605 1416 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842679 1419 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842983 1411 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2ff058e2a2324419b03d708582f033d5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842998 1427 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.846299 1415 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_3e7f39a3248e4963bda4b7790c673d73 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.846441 1409 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_94e1f800f024421e91263edf7388fd9f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)
*** Check failure stack trace: ***
F20260501 13:58:02.840389 1413 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2e04f40a058b4f08a86ed4ca4cd566cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.840448 1420 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.840445 1418 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.841805 1431 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.841912 1407 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b3375377b39846b986b09a1f8fd0b606 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842144 1406 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_215a35e1b7404ba1aa928f075959f1e5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1) 20260501 13:58:02.842183 1414 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b93ea2b7623547349398aaf4f3d85376 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842605 1416 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842679 1419 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842983 1411 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2ff058e2a2324419b03d708582f033d5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842998 1427 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.846299 1415 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_3e7f39a3248e4963bda4b7790c673d73 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.846441 1409 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_94e1f800f024421e91263edf7388fd9f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.846453 1412 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_49594ae63df046b38ccad889763a2723 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)
*** Check failure stack trace: ***
F20260501 13:58:02.840389 1413 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2e04f40a058b4f08a86ed4ca4cd566cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.840448 1420 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.840445 1418 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.841805 1431 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.841912 1407 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b3375377b39846b986b09a1f8fd0b606 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842144 1406 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_215a35e1b7404ba1aa928f075959f1e5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1) 20260501 13:58:02.842183 1414 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b93ea2b7623547349398aaf4f3d85376 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842605 1416 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842679 1419 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842983 1411 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2ff058e2a2324419b03d708582f033d5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842998 1427 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.846299 1415 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_3e7f39a3248e4963bda4b7790c673d73 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.846441 1409 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_94e1f800f024421e91263edf7388fd9f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.846453 1412 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_49594ae63df046b38ccad889763a2723 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.846872 1417 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.846894 *** Check failure stack trace: ***
F20260501 13:58:02.840389 1413 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2e04f40a058b4f08a86ed4ca4cd566cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.840448 1420 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.840445 1418 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.841805 1431 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.841912 1407 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b3375377b39846b986b09a1f8fd0b606 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842144 1406 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_215a35e1b7404ba1aa928f075959f1e5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1) 20260501 13:58:02.842183 1414 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b93ea2b7623547349398aaf4f3d85376 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842605 1416 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842679 1419 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842983 1411 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2ff058e2a2324419b03d708582f033d5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842998 1427 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.846299 1415 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_3e7f39a3248e4963bda4b7790c673d73 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.846441 1409 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_94e1f800f024421e91263edf7388fd9f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.846453 1412 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_49594ae63df046b38ccad889763a2723 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.846872 1417 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.846894 1426 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)
*** Check failure stack trace: ***
F20260501 13:58:02.840389 1413 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2e04f40a058b4f08a86ed4ca4cd566cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.840448 1420 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.840445 1418 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.841805 1431 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.841912 1407 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b3375377b39846b986b09a1f8fd0b606 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842144 1406 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_215a35e1b7404ba1aa928f075959f1e5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1) 20260501 13:58:02.842183 1414 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b93ea2b7623547349398aaf4f3d85376 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842605 1416 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842679 1419 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842983 1411 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2ff058e2a2324419b03d708582f033d5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842998 1427 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.846299 1415 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_3e7f39a3248e4963bda4b7790c673d73 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.846441 1409 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_94e1f800f024421e91263edf7388fd9f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.846453 1412 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_49594ae63df046b38ccad889763a2723 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.846872 1417 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.846894 1426 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.847056 1410 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_7b25f95aeebb4ef29ab513d7047fe028 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)
*** Check failure stack trace: ***
@ 0x7f507a7b0980 (unknown) at ??:0
F20260501 13:58:02.840389 1413 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2e04f40a058b4f08a86ed4ca4cd566cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.840448 1420 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.840445 1418 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.841805 1431 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.841912 1407 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b3375377b39846b986b09a1f8fd0b606 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842144 1406 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_215a35e1b7404ba1aa928f075959f1e5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1) 20260501 13:58:02.842183 1414 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b93ea2b7623547349398aaf4f3d85376 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842605 1416 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842679 1419 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842983 1411 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2ff058e2a2324419b03d708582f033d5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842998 1427 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.846299 1415 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_3e7f39a3248e4963bda4b7790c673d73 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.846441 1409 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_94e1f800f024421e91263edf7388fd9f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.846453 1412 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_49594ae63df046b38ccad889763a2723 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.846872 1417 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.846894 1426 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.847056 1410 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_7b25f95aeebb4ef29ab513d7047fe028 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.848780 1408 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_058c3f74f9b04b9d92dcd4667e109a5d on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F*** Check failure stack trace: ***
F20260501 13:58:02.840389 1413 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2e04f40a058b4f08a86ed4ca4cd566cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.840448 1420 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.840445 1418 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.841805 1431 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.841912 1407 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b3375377b39846b986b09a1f8fd0b606 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842144 1406 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_215a35e1b7404ba1aa928f075959f1e5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1) 20260501 13:58:02.842183 1414 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b93ea2b7623547349398aaf4f3d85376 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842605 1416 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842679 1419 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842983 1411 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2ff058e2a2324419b03d708582f033d5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842998 1427 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.846299 1415 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_3e7f39a3248e4963bda4b7790c673d73 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.846441 1409 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_94e1f800f024421e91263edf7388fd9f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.846453 1412 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_49594ae63df046b38ccad889763a2723 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.846872 1417 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.846894 1426 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.847056 1410 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_7b25f95aeebb4ef29ab513d7047fe028 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.848780 1408 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_058c3f74f9b04b9d92dcd4667e109a5d on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.848820 1430 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)
*** Check failure stack trace: ***
F20260501 13:58:02.840389 1413 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2e04f40a058b4f08a86ed4ca4cd566cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.840448 1420 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.840445 1418 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.841805 1431 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.841912 1407 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b3375377b39846b986b09a1f8fd0b606 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842144 1406 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_215a35e1b7404ba1aa928f075959f1e5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1) 20260501 13:58:02.842183 1414 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b93ea2b7623547349398aaf4f3d85376 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842605 1416 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842679 1419 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842983 1411 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2ff058e2a2324419b03d708582f033d5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.842998 1427 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.846299 1415 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_3e7f39a3248e4963bda4b7790c673d73 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.846441 1409 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_94e1f800f024421e91263edf7388fd9f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.846453 1412 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_49594ae63df046b38ccad889763a2723 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.846872 1417 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.846894 1426 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.847056 1410 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_7b25f95aeebb4ef29ab513d7047fe028 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.848780 1408 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_058c3f74f9b04b9d92dcd4667e109a5d on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.848820 1430 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)F20260501 13:58:02.849000 1428 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 770) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:44705,127.0.105.61:33209,127.0.105.60:39729, num_attempts: 1)
*** Check failure stack trace: ***
@ 0x7f50735a0e87 gsignal at ??:0
@ 0x7f50735a27f1 abort at ??:0
@ 0x7f50762dadcd google::LogMessage::Fail() at ??:0
@ 0x7f50762deb93 google::LogMessage::SendToLog() at ??:0
@ 0x7f50762da7cc google::LogMessage::Flush() at ??:0
@ 0x7f50762dbf59 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x40cb7c kudu::MasterStressTest::DeleteTableThread() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/master-stress-test.cc:351
@ 0x44aaf4 std::__invoke_impl<>() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/invoke.h:73
@ 0x44a9ad std::__invoke<>() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/invoke.h:95
@ 0x44a96a _ZNSt6thread8_InvokerISt5tupleIJMN4kudu16MasterStressTestEFvvEPNS2_26MasterStressTest_Test_TestEEEE9_M_invokeIJLm0ELm1EEEEDTclsr3stdE8__invokespcl10_S_declvalIXT_EEEEESt12_Index_tupleIJXspT_EEE at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:234
@ 0x44a91e std::thread::_Invoker<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:243
@ 0x44a659 std::thread::_State_impl<>::_M_run() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:186
@ 0x7f5073c286df (unknown) at ??:0
@ 0x7f507a7a56db start_thread at ??:0
@ 0x7f507368361f clone at ??:0