Note: This is test shard 3 of 3.
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from CatalogManagerConfigurations/MasterStressTest
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/2
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260807 19:41:48.238794 6580 test_util.cc:268] Using random seed: 838720396
2026-08-07T19:41:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:41:48Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskeMwJpl/thirdparty/src/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskeMwJpl/thirdparty/src/hadoop-3.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/security.properties
2026-08-07 19:41:55: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskeMwJpl/thirdparty/src/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskeMwJpl/thirdparty/src/hadoop-3.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
I20260807 19:42:03.809954 6580 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskeMwJpl/build/asan/bin/kudu
/tmp/dist-test-taskeMwJpl/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.109.62:43417
--webserver_interface=127.6.109.62
--webserver_port=0
--builtin_ntp_servers=127.6.109.20:40195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129
--hive_metastore_uris=thrift://127.0.0.1:38987
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260807 19:42:04.182371 6828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:04.182740 6828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:04.182896 6828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:04.190601 6828 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:04.190725 6828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:04.190814 6828 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:04.190943 6828 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:04.200717 6828 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.109.20:40195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:38987
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.109.62:43417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.109.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:04.202366 6828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:04.204420 6828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:04.220284 6833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:04.220331 6836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:04.220386 6834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:04.223476 6828 server_base.cc:1034] running on GCE node
I20260807 19:42:04.224481 6828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:04.226287 6828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:04.227530 6828 hybrid_clock.cc:648] HybridClock initialized: now 1786131724227440 us; error 128 us; skew 500 ppm
I20260807 19:42:04.227917 6828 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:42:04.230788 6828 webserver.cc:460] Webserver started at http://127.6.109.62:37311/ using document root <none> and password file <none>
I20260807 19:42:04.231594 6828 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:04.231734 6828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:04.232106 6828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:04.235356 6828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/data/instance:
uuid: "959ac7807a8b42a59c61a23d4605c054"
format_stamp: "Formatted at 2026-08-07 19:42:04 on dist-test-slave-p3t5"
I20260807 19:42:04.236284 6828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/wal/instance:
uuid: "959ac7807a8b42a59c61a23d4605c054"
format_stamp: "Formatted at 2026-08-07 19:42:04 on dist-test-slave-p3t5"
I20260807 19:42:04.243077 6828 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260807 19:42:04.247255 6842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:04.249061 6828 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 19:42:04.249323 6828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/data,/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/wal
uuid: "959ac7807a8b42a59c61a23d4605c054"
format_stamp: "Formatted at 2026-08-07 19:42:04 on dist-test-slave-p3t5"
I20260807 19:42:04.249578 6828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:04.264055 6828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:04.265071 6828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:04.265465 6828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:04.287285 6828 rpc_server.cc:307] RPC server started. Bound to: 127.6.109.62:43417
I20260807 19:42:04.287318 6894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.109.62:43417 every 8 connection(s)
I20260807 19:42:04.288936 6828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/data/info.pb
I20260807 19:42:04.292240 6580 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskeMwJpl/build/asan/bin/kudu as pid 6828
I20260807 19:42:04.292598 6580 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/wal/instance
I20260807 19:42:04.293390 6895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:04.295682 6580 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskeMwJpl/build/asan/bin/kudu
/tmp/dist-test-taskeMwJpl/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.109.61:39263
--webserver_interface=127.6.109.61
--webserver_port=0
--builtin_ntp_servers=127.6.109.20:40195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129
--hive_metastore_uris=thrift://127.0.0.1:38987
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260807 19:42:04.300266 6895 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:42:04.323978 6895 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } has no permanent_uuid. Determining permanent_uuid...
W20260807 19:42:04.325501 6846 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:42:04.327419 6895 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.109.61:39263: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:42:04.376914 6895 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } attempt: 1
W20260807 19:42:04.379202 6895 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.109.61:39263: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:42:04.447584 6895 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } attempt: 2
W20260807 19:42:04.450577 6895 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.109.61:39263: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:42:04.541971 6895 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } attempt: 3
W20260807 19:42:04.544440 6895 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.109.61:39263: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:42:04.661325 6896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:04.661767 6896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:04.661873 6896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:04.669495 6896 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:04.669674 6896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:04.669747 6896 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:04.669817 6896 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:04.679378 6896 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.109.20:40195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:38987
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.109.61:39263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.109.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:04.680742 6896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:04.682576 6896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 19:42:04.687734 6895 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } attempt: 4
W20260807 19:42:04.690217 6895 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.109.61:39263: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:42:04.695729 6909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:04.695822 6906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:04.695796 6907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:04.697005 6896 server_base.cc:1034] running on GCE node
I20260807 19:42:04.698024 6896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:04.699503 6896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:04.700779 6896 hybrid_clock.cc:648] HybridClock initialized: now 1786131724700715 us; error 45 us; skew 500 ppm
I20260807 19:42:04.701154 6896 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:42:04.703883 6896 webserver.cc:460] Webserver started at http://127.6.109.61:37257/ using document root <none> and password file <none>
I20260807 19:42:04.704640 6896 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:04.704797 6896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:04.705134 6896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:04.708187 6896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-1/data/instance:
uuid: "1b782e30a8d44e7b94f080aa6327134f"
format_stamp: "Formatted at 2026-08-07 19:42:04 on dist-test-slave-p3t5"
I20260807 19:42:04.709072 6896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-1/wal/instance:
uuid: "1b782e30a8d44e7b94f080aa6327134f"
format_stamp: "Formatted at 2026-08-07 19:42:04 on dist-test-slave-p3t5"
I20260807 19:42:04.715016 6896 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260807 19:42:04.718796 6916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:04.720329 6896 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 19:42:04.720582 6896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-1/data,/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-1/wal
uuid: "1b782e30a8d44e7b94f080aa6327134f"
format_stamp: "Formatted at 2026-08-07 19:42:04 on dist-test-slave-p3t5"
I20260807 19:42:04.720822 6896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:04.737344 6896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:04.738348 6896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:04.738700 6896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:04.759809 6896 rpc_server.cc:307] RPC server started. Bound to: 127.6.109.61:39263
I20260807 19:42:04.759843 6968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.109.61:39263 every 8 connection(s)
I20260807 19:42:04.761334 6896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-1/data/info.pb
I20260807 19:42:04.765436 6969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:04.767935 6580 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskeMwJpl/build/asan/bin/kudu as pid 6896
I20260807 19:42:04.768127 6580 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-1/wal/instance
I20260807 19:42:04.769579 6580 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskeMwJpl/build/asan/bin/kudu
/tmp/dist-test-taskeMwJpl/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.109.60:43129
--webserver_interface=127.6.109.60
--webserver_port=0
--builtin_ntp_servers=127.6.109.20:40195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129
--hive_metastore_uris=thrift://127.0.0.1:38987
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260807 19:42:04.775432 6969 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:42:04.792440 6969 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:42:04.804172 6969 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } has no permanent_uuid. Determining permanent_uuid...
W20260807 19:42:04.805397 6920 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: connect: Connection refused (error 111)
W20260807 19:42:04.807248 6969 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.109.60:43129: Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: connect: Connection refused (error 111)
I20260807 19:42:04.856649 6969 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } attempt: 1
W20260807 19:42:04.858995 6969 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.109.60:43129: Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: connect: Connection refused (error 111)
I20260807 19:42:04.927310 6969 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } attempt: 2
W20260807 19:42:04.929544 6969 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.109.60:43129: Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: connect: Connection refused (error 111)
I20260807 19:42:04.989498 6895 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } attempt: 5
I20260807 19:42:04.998246 6895 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } has no permanent_uuid. Determining permanent_uuid...
W20260807 19:42:05.000350 6895 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.109.60:43129: Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: connect: Connection refused (error 111)
I20260807 19:42:05.020867 6969 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } attempt: 3
W20260807 19:42:05.023398 6969 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.109.60:43129: Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: connect: Connection refused (error 111)
I20260807 19:42:05.051647 6895 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } attempt: 1
W20260807 19:42:05.054337 6895 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.109.60:43129: Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: connect: Connection refused (error 111)
I20260807 19:42:05.122678 6895 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } attempt: 2
W20260807 19:42:05.125203 6895 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.109.60:43129: Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: connect: Connection refused (error 111)
W20260807 19:42:05.139012 6970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:05.139346 6970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:05.139488 6970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:05.147214 6970 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:05.147346 6970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:05.147389 6970 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:05.147480 6970 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:05.157239 6970 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.109.20:40195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:38987
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.109.60:43129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.109.60
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:05.158815 6970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:05.160746 6970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 19:42:05.166671 6969 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } attempt: 4
W20260807 19:42:05.168960 6969 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.109.60:43129: Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: connect: Connection refused (error 111)
W20260807 19:42:05.174007 6978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:05.173969 6981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:05.173885 6979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:05.175021 6970 server_base.cc:1034] running on GCE node
I20260807 19:42:05.175940 6970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:05.177476 6970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:05.178732 6970 hybrid_clock.cc:648] HybridClock initialized: now 1786131725178647 us; error 85 us; skew 500 ppm
I20260807 19:42:05.179104 6970 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:42:05.181756 6970 webserver.cc:460] Webserver started at http://127.6.109.60:36383/ using document root <none> and password file <none>
I20260807 19:42:05.182516 6970 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:05.182674 6970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:05.183022 6970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:05.185925 6970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data/instance:
uuid: "7b31edda0c4a48489ad60fc59c44ed31"
format_stamp: "Formatted at 2026-08-07 19:42:05 on dist-test-slave-p3t5"
I20260807 19:42:05.186786 6970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/wal/instance:
uuid: "7b31edda0c4a48489ad60fc59c44ed31"
format_stamp: "Formatted at 2026-08-07 19:42:05 on dist-test-slave-p3t5"
I20260807 19:42:05.192459 6970 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:42:05.196233 6987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:05.197810 6970 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:42:05.198030 6970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data,/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/wal
uuid: "7b31edda0c4a48489ad60fc59c44ed31"
format_stamp: "Formatted at 2026-08-07 19:42:05 on dist-test-slave-p3t5"
I20260807 19:42:05.198261 6970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:05.221064 6970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:05.222128 6970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:05.222481 6970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:05.231508 6895 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } attempt: 3
W20260807 19:42:05.234025 6895 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.109.60:43129: Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: connect: Connection refused (error 111)
I20260807 19:42:05.243300 6970 rpc_server.cc:307] RPC server started. Bound to: 127.6.109.60:43129
I20260807 19:42:05.243324 7038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.109.60:43129 every 8 connection(s)
I20260807 19:42:05.244804 6970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data/info.pb
I20260807 19:42:05.248890 7039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:05.252348 6580 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskeMwJpl/build/asan/bin/kudu as pid 6970
I20260807 19:42:05.252564 6580 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/wal/instance
I20260807 19:42:05.257171 7039 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:42:05.276528 7039 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:42:05.284971 7039 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:42:05.308029 7039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: Bootstrap starting.
I20260807 19:42:05.311684 7039 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:05.312788 7039 log.cc:824] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:05.315861 7039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: No bootstrap required, opened a new log
I20260807 19:42:05.323953 7039 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:05.324865 7039 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:05.325050 7039 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b31edda0c4a48489ad60fc59c44ed31, State: Initialized, Role: FOLLOWER
I20260807 19:42:05.325954 7039 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:05.326988 7047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:05.327335 7047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:05.328110 7039 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 19:42:05.343636 7061 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 19:42:05.343950 7061 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 19:42:05.340728 6970 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: OK, instance_id { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" instance_seqno: 1786131724280891 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 19:42:05.354550 6970 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:42:05.411302 6895 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } attempt: 4
I20260807 19:42:05.426028 6895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054: Bootstrap starting.
I20260807 19:42:05.430743 6895 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:05.432322 6895 log.cc:824] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:05.435851 6895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054: No bootstrap required, opened a new log
I20260807 19:42:05.447724 6895 raft_consensus.cc:348] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:05.448220 6895 raft_consensus.cc:374] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:05.448392 6895 raft_consensus.cc:729] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 959ac7807a8b42a59c61a23d4605c054, State: Initialized, Role: FOLLOWER
I20260807 19:42:05.449177 6895 consensus_queue.cc:260] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:05.450273 7065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:05.450655 7065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:05.451797 6895 sys_catalog.cc:564] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 19:42:05.466040 7079 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 19:42:05.466295 7079 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 19:42:05.468211 6969 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } attempt: 5
W20260807 19:42:05.474737 6993 tablet.cc:2369] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:42:05.465638 6828 master_runner.cc:186] Error getting master registration for 127.6.109.61:39263: OK, instance_id { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" instance_seqno: 1786131724753214 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 19:42:05.477157 6828 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:42:05.483174 6969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f: Bootstrap starting.
I20260807 19:42:05.487824 6969 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:05.489444 6969 log.cc:824] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:05.499682 6969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f: No bootstrap required, opened a new log
I20260807 19:42:05.511969 6969 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:05.512598 6969 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:05.512787 6969 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b782e30a8d44e7b94f080aa6327134f, State: Initialized, Role: FOLLOWER
I20260807 19:42:05.513767 6969 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
W20260807 19:42:05.519511 6848 tablet.cc:2369] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:42:05.519950 6969 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:05.523705 7083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:05.524273 7083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:05.550508 6896 master_runner.cc:263] No leader master found from master 1b782e30a8d44e7b94f080aa6327134f
I20260807 19:42:05.550678 6896 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 19:42:05.569413 7100 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 19:42:05.569721 7100 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 19:42:05.582870 7047 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:05.583256 7047 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:05.585063 7047 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
I20260807 19:42:05.586968 6869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7b31edda0c4a48489ad60fc59c44ed31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "959ac7807a8b42a59c61a23d4605c054" is_pre_election: true
I20260807 19:42:05.587734 6869 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b31edda0c4a48489ad60fc59c44ed31 in term 0.
I20260807 19:42:05.587610 6943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7b31edda0c4a48489ad60fc59c44ed31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b782e30a8d44e7b94f080aa6327134f" is_pre_election: true
I20260807 19:42:05.588187 6943 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b31edda0c4a48489ad60fc59c44ed31 in term 0.
I20260807 19:42:05.588721 6990 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31, 959ac7807a8b42a59c61a23d4605c054; no voters:
I20260807 19:42:05.589913 7047 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:42:05.590077 7047 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:42:05.590188 7047 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:05.598136 7047 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:05.602175 6869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7b31edda0c4a48489ad60fc59c44ed31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "959ac7807a8b42a59c61a23d4605c054"
I20260807 19:42:05.602480 6869 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:05.606369 6869 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b31edda0c4a48489ad60fc59c44ed31 in term 1.
I20260807 19:42:05.607106 6990 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31, 959ac7807a8b42a59c61a23d4605c054; no voters:
I20260807 19:42:05.611371 6943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7b31edda0c4a48489ad60fc59c44ed31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b782e30a8d44e7b94f080aa6327134f"
I20260807 19:42:05.611685 6943 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:05.615518 6943 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b31edda0c4a48489ad60fc59c44ed31 in term 1.
I20260807 19:42:05.616566 7047 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 1 election: Requested vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
I20260807 19:42:05.617092 7047 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:05.617607 7047 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 1 LEADER]: Becoming Leader. State: Replica: 7b31edda0c4a48489ad60fc59c44ed31, State: Running, Role: LEADER
I20260807 19:42:05.618571 7047 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:05.643833 7103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b31edda0c4a48489ad60fc59c44ed31. Latest consensus state: current_term: 1 leader_uuid: "7b31edda0c4a48489ad60fc59c44ed31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:05.644347 7103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:05.655547 7105 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:05.658241 7105 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:05.679510 6943 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 1 FOLLOWER]: Refusing update from remote peer 7b31edda0c4a48489ad60fc59c44ed31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:42:05.680712 7103 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:05.688536 6869 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 1 FOLLOWER]: Refusing update from remote peer 7b31edda0c4a48489ad60fc59c44ed31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:42:05.689826 7103 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:05.727810 7065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b31edda0c4a48489ad60fc59c44ed31. Latest consensus state: current_term: 1 leader_uuid: "7b31edda0c4a48489ad60fc59c44ed31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:05.728183 7065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:05.731956 7106 mvcc.cc:205] Tried to move back new op lower bound from 7315995548369567744 to 7315995548165263360. Current Snapshot: MvccSnapshot[applied={T|T < 7315995548369567744}]
I20260807 19:42:05.733080 7108 mvcc.cc:205] Tried to move back new op lower bound from 7315995548369567744 to 7315995548165263360. Current Snapshot: MvccSnapshot[applied={T|T < 7315995548369567744}]
I20260807 19:42:05.734030 7083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b31edda0c4a48489ad60fc59c44ed31. Latest consensus state: current_term: 1 leader_uuid: "7b31edda0c4a48489ad60fc59c44ed31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:05.734375 7083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:05.738615 7107 mvcc.cc:205] Tried to move back new op lower bound from 7315995548369567744 to 7315995548165263360. Current Snapshot: MvccSnapshot[applied={T|T < 7315995548369567744}]
I20260807 19:42:05.745101 7103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7b31edda0c4a48489ad60fc59c44ed31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:05.745445 7103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:05.745971 7103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7b31edda0c4a48489ad60fc59c44ed31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:05.746224 7103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:05.750814 7065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7b31edda0c4a48489ad60fc59c44ed31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:05.751138 7065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:05.752653 7083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7b31edda0c4a48489ad60fc59c44ed31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:05.752966 7083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:05.755074 7105 catalog_manager.cc:1347] Generated new cluster ID: a80f7b000fe149f3a73e545a28c78fda
I20260807 19:42:05.755196 7105 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:05.800635 7105 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:42:05.803009 7105 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:05.852438 7105 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: Generated new TSK 0
I20260807 19:42:05.854578 7105 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:42:05.856897 7105 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260807 19:42:05.868976 7105 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260807 19:42:05.909039 6580 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskeMwJpl/build/asan/bin/kudu
/tmp/dist-test-taskeMwJpl/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.109.1:0
--local_ip_for_outbound_sockets=127.6.109.1
--webserver_interface=127.6.109.1
--webserver_port=0
--tserver_master_addrs=127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129
--builtin_ntp_servers=127.6.109.20:40195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260807 19:42:06.358655 7116 raft_consensus.cc:482] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7b31edda0c4a48489ad60fc59c44ed31)
I20260807 19:42:06.365360 7116 raft_consensus.cc:504] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:06.367705 7115 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7b31edda0c4a48489ad60fc59c44ed31)
I20260807 19:42:06.368021 7115 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:06.370474 6868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1b782e30a8d44e7b94f080aa6327134f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "959ac7807a8b42a59c61a23d4605c054" is_pre_election: true
I20260807 19:42:06.370810 6868 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b782e30a8d44e7b94f080aa6327134f in term 1.
I20260807 19:42:06.371407 7115 leader_election.cc:288] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129)
I20260807 19:42:06.372089 6919 leader_election.cc:302] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054; no voters:
I20260807 19:42:06.372131 7014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1b782e30a8d44e7b94f080aa6327134f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7b31edda0c4a48489ad60fc59c44ed31" is_pre_election: true
I20260807 19:42:06.373900 7115 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:42:06.374352 6943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "959ac7807a8b42a59c61a23d4605c054" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1b782e30a8d44e7b94f080aa6327134f" is_pre_election: true
I20260807 19:42:06.375445 7014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "959ac7807a8b42a59c61a23d4605c054" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7b31edda0c4a48489ad60fc59c44ed31" is_pre_election: true
I20260807 19:42:06.375785 7115 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7b31edda0c4a48489ad60fc59c44ed31)
I20260807 19:42:06.376142 7116 leader_election.cc:288] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263), 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129)
I20260807 19:42:06.376400 7115 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:42:06.382036 7115 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:06.383255 6943 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 959ac7807a8b42a59c61a23d4605c054 in current term 2: Already voted for candidate 1b782e30a8d44e7b94f080aa6327134f in this term.
I20260807 19:42:06.383785 7115 leader_election.cc:288] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [CANDIDATE]: Term 2 election: Requested vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129)
I20260807 19:42:06.384439 6868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1b782e30a8d44e7b94f080aa6327134f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "959ac7807a8b42a59c61a23d4605c054"
I20260807 19:42:06.384721 6868 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:42:06.385222 7014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1b782e30a8d44e7b94f080aa6327134f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7b31edda0c4a48489ad60fc59c44ed31"
I20260807 19:42:06.387341 6846 leader_election.cc:302] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 959ac7807a8b42a59c61a23d4605c054; no voters: 1b782e30a8d44e7b94f080aa6327134f, 7b31edda0c4a48489ad60fc59c44ed31
I20260807 19:42:06.388149 6868 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b782e30a8d44e7b94f080aa6327134f in term 2.
I20260807 19:42:06.388687 6919 leader_election.cc:302] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054; no voters: 7b31edda0c4a48489ad60fc59c44ed31
I20260807 19:42:06.388994 7116 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:42:06.389302 7115 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:42:06.390237 7115 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 2 LEADER]: Becoming Leader. State: Replica: 1b782e30a8d44e7b94f080aa6327134f, State: Running, Role: LEADER
I20260807 19:42:06.390990 7115 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:06.393138 7117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b782e30a8d44e7b94f080aa6327134f. Latest consensus state: current_term: 2 leader_uuid: "1b782e30a8d44e7b94f080aa6327134f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:06.393743 7117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:06.395044 7118 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:06.398312 7118 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:06.400099 7118 catalog_manager.cc:1259] Loaded cluster ID: a80f7b000fe149f3a73e545a28c78fda
I20260807 19:42:06.400298 7118 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:06.404516 7118 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:06.406378 7118 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f: Loaded TSK: 0
I20260807 19:42:06.408133 7118 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:42:06.409591 7118 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260807 19:42:06.411355 7118 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260807 19:42:06.435936 6943 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 2 LEADER]: Rejecting Update request from peer 7b31edda0c4a48489ad60fc59c44ed31 for earlier term 1. Current term is 2. Ops: []
I20260807 19:42:06.436094 6868 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 2 FOLLOWER]: Rejecting Update request from peer 7b31edda0c4a48489ad60fc59c44ed31 for earlier term 1. Current term is 2. Ops: []
I20260807 19:42:06.436794 7120 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 19:42:06.437927 7120 consensus_queue.cc:1225] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [LEADER]: Peer 1b782e30a8d44e7b94f080aa6327134f log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260807 19:42:06.438258 7119 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 19:42:06.438746 7121 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:42:06.439055 7121 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7b31edda0c4a48489ad60fc59c44ed31, State: Running, Role: LEADER
I20260807 19:42:06.439677 7121 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:06.440299 7121 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:42:06.469148 7079 catalog_manager.cc:1259] Loaded cluster ID: a80f7b000fe149f3a73e545a28c78fda
I20260807 19:42:06.469300 7079 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054: loading cluster ID for follower catalog manager: success
I20260807 19:42:06.472886 7079 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054: acquiring CA information for follower catalog manager: success
I20260807 19:42:06.475462 7079 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 19:42:06.674044 7114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:06.674494 7114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:06.674768 7114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:06.683785 7114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:06.684157 7114 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.109.1
I20260807 19:42:06.696180 7114 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.109.20:40195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.109.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.109.1
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.109.1
--log_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:06.697952 7114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:06.700314 7114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:06.718708 7126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:06.718986 7127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:06.719039 7130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:06.720479 7114 server_base.cc:1034] running on GCE node
I20260807 19:42:06.721263 7114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:06.722374 7114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:06.723697 7114 hybrid_clock.cc:648] HybridClock initialized: now 1786131726723557 us; error 53 us; skew 500 ppm
I20260807 19:42:06.724108 7114 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:42:06.727977 7114 webserver.cc:460] Webserver started at http://127.6.109.1:35581/ using document root <none> and password file <none>
I20260807 19:42:06.729034 7114 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:06.729281 7114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:06.729779 7114 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:06.733502 7114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-0/data/instance:
uuid: "051d464d644a4d4486462dfb18274c6a"
format_stamp: "Formatted at 2026-08-07 19:42:06 on dist-test-slave-p3t5"
I20260807 19:42:06.734591 7114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-0/wal/instance:
uuid: "051d464d644a4d4486462dfb18274c6a"
format_stamp: "Formatted at 2026-08-07 19:42:06 on dist-test-slave-p3t5"
I20260807 19:42:06.742118 7114 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260807 19:42:06.747200 7136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:06.749189 7114 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260807 19:42:06.749498 7114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-0/data,/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-0/wal
uuid: "051d464d644a4d4486462dfb18274c6a"
format_stamp: "Formatted at 2026-08-07 19:42:06 on dist-test-slave-p3t5"
I20260807 19:42:06.749923 7114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:06.772085 7114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:06.773751 7114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:06.774282 7114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:06.775878 7114 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:06.778316 7114 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:06.778491 7114 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:06.778744 7114 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:06.778913 7114 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:06.847462 7121 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:06.847802 7121 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:06.848719 7121 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
I20260807 19:42:06.849668 6868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7b31edda0c4a48489ad60fc59c44ed31" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "959ac7807a8b42a59c61a23d4605c054" is_pre_election: true
I20260807 19:42:06.849782 6943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7b31edda0c4a48489ad60fc59c44ed31" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1b782e30a8d44e7b94f080aa6327134f" is_pre_election: true
I20260807 19:42:06.850008 6868 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b31edda0c4a48489ad60fc59c44ed31 in term 2.
I20260807 19:42:06.851015 6990 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31, 959ac7807a8b42a59c61a23d4605c054; no voters:
I20260807 19:42:06.851509 7121 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:42:06.851722 7121 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:42:06.851895 7121 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:42:06.855654 7121 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:06.857303 6868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7b31edda0c4a48489ad60fc59c44ed31" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "959ac7807a8b42a59c61a23d4605c054"
I20260807 19:42:06.857621 6868 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:42:06.858961 7114 rpc_server.cc:307] RPC server started. Bound to: 127.6.109.1:44467
I20260807 19:42:06.859077 7254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.109.1:44467 every 8 connection(s)
I20260807 19:42:06.860466 7114 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-0/data/info.pb
I20260807 19:42:06.860780 6868 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b31edda0c4a48489ad60fc59c44ed31 in term 3.
I20260807 19:42:06.861387 6990 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31, 959ac7807a8b42a59c61a23d4605c054; no voters:
I20260807 19:42:06.861768 7119 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:42:06.862006 7119 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 3 LEADER]: Becoming Leader. State: Replica: 7b31edda0c4a48489ad60fc59c44ed31, State: Running, Role: LEADER
I20260807 19:42:06.862226 6943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7b31edda0c4a48489ad60fc59c44ed31" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1b782e30a8d44e7b94f080aa6327134f"
I20260807 19:42:06.862366 7119 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:06.863044 7121 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 3 election: Requested vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
I20260807 19:42:06.864120 6580 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskeMwJpl/build/asan/bin/kudu as pid 7114
I20260807 19:42:06.864428 6580 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-0/wal/instance
I20260807 19:42:06.875622 6868 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 3 FOLLOWER]: Rejecting Update request from peer 1b782e30a8d44e7b94f080aa6327134f for earlier term 2. Current term is 3. Ops: []
I20260807 19:42:06.876585 7117 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 19:42:06.876824 6580 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskeMwJpl/build/asan/bin/kudu
/tmp/dist-test-taskeMwJpl/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.109.2:0
--local_ip_for_outbound_sockets=127.6.109.2
--webserver_interface=127.6.109.2
--webserver_port=0
--tserver_master_addrs=127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129
--builtin_ntp_servers=127.6.109.20:40195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260807 19:42:06.877173 7115 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:42:06.877364 7115 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1b782e30a8d44e7b94f080aa6327134f, State: Running, Role: LEADER
I20260807 19:42:06.877736 7115 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:06.878599 7115 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:42:06.893301 7121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b31edda0c4a48489ad60fc59c44ed31. Latest consensus state: current_term: 3 leader_uuid: "7b31edda0c4a48489ad60fc59c44ed31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:06.893787 7121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:06.908718 7265 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:06.915422 7256 heartbeater.cc:347] Connected to a master server at 127.6.109.62:43417
I20260807 19:42:06.915874 7256 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:06.917371 7256 heartbeater.cc:511] Master 127.6.109.62:43417 requested a full tablet report, sending...
I20260807 19:42:06.918078 7265 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:06.919972 7265 catalog_manager.cc:1259] Loaded cluster ID: a80f7b000fe149f3a73e545a28c78fda
I20260807 19:42:06.920383 7265 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:06.920665 6859 ts_manager.cc:194] Registered new tserver with Master: 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467)
I20260807 19:42:06.921644 7255 heartbeater.cc:347] Connected to a master server at 127.6.109.60:43129
I20260807 19:42:06.921845 7255 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:06.922690 7255 heartbeater.cc:511] Master 127.6.109.60:43129 requested a full tablet report, sending...
I20260807 19:42:06.924942 7004 ts_manager.cc:194] Registered new tserver with Master: 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467)
I20260807 19:42:06.925733 7265 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:06.930002 7265 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: Loaded TSK: 0
I20260807 19:42:06.931427 7265 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:42:06.932799 7265 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260807 19:42:06.934513 7265 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260807 19:42:06.936371 7262 heartbeater.cc:347] Connected to a master server at 127.6.109.61:39263
I20260807 19:42:06.936594 7262 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:06.937100 7262 heartbeater.cc:511] Master 127.6.109.61:39263 requested a full tablet report, sending...
I20260807 19:42:06.939252 6933 ts_manager.cc:194] Registered new tserver with Master: 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467)
I20260807 19:42:07.298998 7115 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:07.299345 7115 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:07.300133 7115 leader_election.cc:288] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129)
I20260807 19:42:07.300928 6868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1b782e30a8d44e7b94f080aa6327134f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "959ac7807a8b42a59c61a23d4605c054" is_pre_election: true
I20260807 19:42:07.300925 7014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1b782e30a8d44e7b94f080aa6327134f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7b31edda0c4a48489ad60fc59c44ed31" is_pre_election: true
I20260807 19:42:07.301260 6868 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b782e30a8d44e7b94f080aa6327134f in term 3.
I20260807 19:42:07.301849 6919 leader_election.cc:302] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054; no voters: 7b31edda0c4a48489ad60fc59c44ed31
I20260807 19:42:07.302320 7115 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:42:07.302467 7115 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:42:07.302764 7115 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:42:07.305279 7115 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:07.305968 7115 leader_election.cc:288] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [CANDIDATE]: Term 4 election: Requested vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129)
I20260807 19:42:07.306694 7014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1b782e30a8d44e7b94f080aa6327134f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7b31edda0c4a48489ad60fc59c44ed31"
I20260807 19:42:07.306741 6868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1b782e30a8d44e7b94f080aa6327134f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "959ac7807a8b42a59c61a23d4605c054"
I20260807 19:42:07.307102 6868 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:42:07.311120 6868 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b782e30a8d44e7b94f080aa6327134f in term 4.
I20260807 19:42:07.311820 6919 leader_election.cc:302] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054; no voters: 7b31edda0c4a48489ad60fc59c44ed31
I20260807 19:42:07.312286 7115 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 4 FOLLOWER]: Leader election won for term 4
I20260807 19:42:07.312661 7115 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 4 LEADER]: Becoming Leader. State: Replica: 1b782e30a8d44e7b94f080aa6327134f, State: Running, Role: LEADER
I20260807 19:42:07.313069 7115 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:07.314567 7117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b782e30a8d44e7b94f080aa6327134f. Latest consensus state: current_term: 4 leader_uuid: "1b782e30a8d44e7b94f080aa6327134f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:07.315038 7117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:07.315932 7268 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:07.318127 7268 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:07.319136 7268 catalog_manager.cc:1259] Loaded cluster ID: a80f7b000fe149f3a73e545a28c78fda
I20260807 19:42:07.319293 7268 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:07.321213 7268 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:07.322358 7268 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f: Loaded TSK: 0
I20260807 19:42:07.322999 7268 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:42:07.324147 7268 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260807 19:42:07.325543 7268 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260807 19:42:07.351063 6943 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 4 LEADER]: Rejecting Update request from peer 7b31edda0c4a48489ad60fc59c44ed31 for earlier term 3. Current term is 4. Ops: []
I20260807 19:42:07.352269 7121 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 19:42:07.352653 7121 consensus_queue.cc:1225] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [LEADER]: Peer 1b782e30a8d44e7b94f080aa6327134f log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260807 19:42:07.352981 7119 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:42:07.353139 7119 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7b31edda0c4a48489ad60fc59c44ed31, State: Running, Role: LEADER
I20260807 19:42:07.353523 7119 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:07.353938 7119 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:42:07.444092 6933 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.109.1:44365
W20260807 19:42:07.467394 7263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:07.467777 7263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:07.467957 7263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:07.477908 7263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:07.478143 7263 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.109.2
I20260807 19:42:07.489298 7263 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.109.20:40195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.109.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.109.2
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.109.2
--log_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:07.491145 7263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:07.493336 7263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:07.514282 7273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:07.515223 7276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:07.515502 7274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:07.516654 7263 server_base.cc:1034] running on GCE node
I20260807 19:42:07.517551 7263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:07.519018 7263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:07.520463 7263 hybrid_clock.cc:648] HybridClock initialized: now 1786131727520255 us; error 64 us; skew 500 ppm
I20260807 19:42:07.520925 7263 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:42:07.524009 7263 webserver.cc:460] Webserver started at http://127.6.109.2:34113/ using document root <none> and password file <none>
I20260807 19:42:07.524919 7263 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:07.525081 7263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:07.525540 7263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:07.529053 7263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-1/data/instance:
uuid: "35b5ecc2c9ef47c2bbe35653633e14da"
format_stamp: "Formatted at 2026-08-07 19:42:07 on dist-test-slave-p3t5"
I20260807 19:42:07.530222 7263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-1/wal/instance:
uuid: "35b5ecc2c9ef47c2bbe35653633e14da"
format_stamp: "Formatted at 2026-08-07 19:42:07 on dist-test-slave-p3t5"
I20260807 19:42:07.537822 7263 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260807 19:42:07.542667 7282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:07.544518 7263 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 19:42:07.544780 7263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-1/data,/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-1/wal
uuid: "35b5ecc2c9ef47c2bbe35653633e14da"
format_stamp: "Formatted at 2026-08-07 19:42:07 on dist-test-slave-p3t5"
I20260807 19:42:07.545022 7263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:07.563071 7263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:07.564455 7263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:07.564829 7263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:07.566432 7263 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:07.568864 7263 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:07.569051 7263 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:07.569305 7263 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:07.569473 7263 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:07.644253 7263 rpc_server.cc:307] RPC server started. Bound to: 127.6.109.2:35483
I20260807 19:42:07.644248 7396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.109.2:35483 every 8 connection(s)
I20260807 19:42:07.646158 7263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-1/data/info.pb
I20260807 19:42:07.653383 6580 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskeMwJpl/build/asan/bin/kudu as pid 7263
I20260807 19:42:07.653734 6580 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-1/wal/instance
I20260807 19:42:07.664669 6580 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskeMwJpl/build/asan/bin/kudu
/tmp/dist-test-taskeMwJpl/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.109.3:0
--local_ip_for_outbound_sockets=127.6.109.3
--webserver_interface=127.6.109.3
--webserver_port=0
--tserver_master_addrs=127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129
--builtin_ntp_servers=127.6.109.20:40195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260807 19:42:07.688102 7398 heartbeater.cc:347] Connected to a master server at 127.6.109.62:43417
I20260807 19:42:07.688122 7397 heartbeater.cc:347] Connected to a master server at 127.6.109.60:43129
I20260807 19:42:07.688221 7399 heartbeater.cc:347] Connected to a master server at 127.6.109.61:39263
I20260807 19:42:07.688654 7397 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:07.688678 7399 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:07.688719 7398 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:07.690728 7398 heartbeater.cc:511] Master 127.6.109.62:43417 requested a full tablet report, sending...
I20260807 19:42:07.690850 7397 heartbeater.cc:511] Master 127.6.109.60:43129 requested a full tablet report, sending...
I20260807 19:42:07.690884 7399 heartbeater.cc:511] Master 127.6.109.61:39263 requested a full tablet report, sending...
I20260807 19:42:07.693164 6933 ts_manager.cc:194] Registered new tserver with Master: 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483)
I20260807 19:42:07.693302 7004 ts_manager.cc:194] Registered new tserver with Master: 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483)
I20260807 19:42:07.694509 6859 ts_manager.cc:194] Registered new tserver with Master: 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483)
I20260807 19:42:07.694682 6933 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.109.2:59751
I20260807 19:42:07.719538 7014 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 4 FOLLOWER]: Refusing update from remote peer 1b782e30a8d44e7b94f080aa6327134f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260807 19:42:07.720664 7117 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 19:42:07.721309 7117 consensus_queue.cc:1225] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [LEADER]: Peer 7b31edda0c4a48489ad60fc59c44ed31 log is divergent from this leader: its last log entry 3.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260807 19:42:07.723070 7014 pending_rounds.cc:76] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: Aborting all ops after (but not including) 4
I20260807 19:42:07.723191 7014 pending_rounds.cc:97] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260807 19:42:07.723516 7014 raft_consensus.cc:2866] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:42:07.725034 7119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b782e30a8d44e7b94f080aa6327134f. Latest consensus state: current_term: 4 leader_uuid: "1b782e30a8d44e7b94f080aa6327134f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:07.725462 7119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:07.728444 6868 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 4 FOLLOWER]: Refusing update from remote peer 1b782e30a8d44e7b94f080aa6327134f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260807 19:42:07.728121 7115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "1b782e30a8d44e7b94f080aa6327134f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:07.728787 7115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:07.729036 7119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "1b782e30a8d44e7b94f080aa6327134f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:07.729399 7119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:07.729511 7115 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [LEADER]: Connected to new peer: Peer: permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 19:42:07.729889 7121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "1b782e30a8d44e7b94f080aa6327134f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:07.730216 7121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:07.739709 7415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "1b782e30a8d44e7b94f080aa6327134f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:07.740799 7415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:07.740173 7411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b782e30a8d44e7b94f080aa6327134f. Latest consensus state: current_term: 4 leader_uuid: "1b782e30a8d44e7b94f080aa6327134f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:07.741357 7411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:07.741488 7414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "1b782e30a8d44e7b94f080aa6327134f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:07.742465 7414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:07.947921 7262 heartbeater.cc:503] Master 127.6.109.61:39263 was elected leader, sending a full tablet report...
I20260807 19:42:08.198091 7399 heartbeater.cc:503] Master 127.6.109.61:39263 was elected leader, sending a full tablet report...
I20260807 19:42:08.227406 7121 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1b782e30a8d44e7b94f080aa6327134f)
I20260807 19:42:08.228049 7121 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:08.228668 7115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "1b782e30a8d44e7b94f080aa6327134f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:08.228973 7121 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
I20260807 19:42:08.229102 7115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:08.230551 6943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7b31edda0c4a48489ad60fc59c44ed31" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1b782e30a8d44e7b94f080aa6327134f" is_pre_election: true
I20260807 19:42:08.231334 6869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7b31edda0c4a48489ad60fc59c44ed31" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "959ac7807a8b42a59c61a23d4605c054" is_pre_election: true
I20260807 19:42:08.232137 6990 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:42:08.232450 7121 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260807 19:42:08.256155 7407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:08.256635 7407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:08.256839 7407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:08.266649 7407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:08.266891 7407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.109.3
I20260807 19:42:08.278245 7407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.109.20:40195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.109.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.109.3
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.109.3
--log_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:08.279861 7407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:08.282480 7407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:08.304418 7422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:08.304517 7426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:08.308507 7423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:08.309226 7407 server_base.cc:1034] running on GCE node
I20260807 19:42:08.310343 7407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:08.311914 7407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:08.313447 7407 hybrid_clock.cc:648] HybridClock initialized: now 1786131728313196 us; error 55 us; skew 500 ppm
I20260807 19:42:08.314002 7407 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:42:08.318105 7407 webserver.cc:460] Webserver started at http://127.6.109.3:40421/ using document root <none> and password file <none>
I20260807 19:42:08.319126 7407 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:08.319377 7407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:08.319769 7407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:08.323369 7407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-2/data/instance:
uuid: "bafdff0fd60146539631a476c4257e16"
format_stamp: "Formatted at 2026-08-07 19:42:08 on dist-test-slave-p3t5"
I20260807 19:42:08.324342 7407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-2/wal/instance:
uuid: "bafdff0fd60146539631a476c4257e16"
format_stamp: "Formatted at 2026-08-07 19:42:08 on dist-test-slave-p3t5"
I20260807 19:42:08.332086 7407 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260807 19:42:08.337426 7432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:08.339203 7407 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:42:08.339373 7407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-2/data,/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-2/wal
uuid: "bafdff0fd60146539631a476c4257e16"
format_stamp: "Formatted at 2026-08-07 19:42:08 on dist-test-slave-p3t5"
I20260807 19:42:08.339561 7407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:08.348225 7061 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 19:42:08.363313 7407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:08.364936 7407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:08.365509 7407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:08.367473 7407 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:08.370186 7407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:08.370326 7407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:08.370504 7407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:08.370656 7407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:08.449342 7407 rpc_server.cc:307] RPC server started. Bound to: 127.6.109.3:40225
I20260807 19:42:08.449421 7545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.109.3:40225 every 8 connection(s)
I20260807 19:42:08.451345 7407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-2/data/info.pb
I20260807 19:42:08.460129 6580 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskeMwJpl/build/asan/bin/kudu as pid 7407
I20260807 19:42:08.460431 6580 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-2/wal/instance
I20260807 19:42:08.483311 7547 heartbeater.cc:347] Connected to a master server at 127.6.109.62:43417
I20260807 19:42:08.483880 7547 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:08.486099 7547 heartbeater.cc:511] Master 127.6.109.62:43417 requested a full tablet report, sending...
I20260807 19:42:08.488039 7546 heartbeater.cc:347] Connected to a master server at 127.6.109.60:43129
I20260807 19:42:08.488317 7546 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:08.488471 6859 ts_manager.cc:194] Registered new tserver with Master: bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
I20260807 19:42:08.488934 7546 heartbeater.cc:511] Master 127.6.109.60:43129 requested a full tablet report, sending...
I20260807 19:42:08.490374 7004 ts_manager.cc:194] Registered new tserver with Master: bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
I20260807 19:42:08.490653 7548 heartbeater.cc:347] Connected to a master server at 127.6.109.61:39263
I20260807 19:42:08.490877 7548 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:08.491465 7548 heartbeater.cc:511] Master 127.6.109.61:39263 requested a full tablet report, sending...
I20260807 19:42:08.492525 6933 ts_manager.cc:194] Registered new tserver with Master: bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
I20260807 19:42:08.493538 6933 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.109.3:43605
I20260807 19:42:08.501196 6580 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:42:08.542450 6933 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41774:
name: "default.Table_93e5970d7fe54e40903e703a8eecd0cf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:08.566457 6927 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41774:
name: "default.Table_ca953b708a5a408898f72c483fc31ecf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:08.567139 6929 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41774:
name: "default.Table_472c6a33c87c4f1b9cdb926a2e5a6344"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:08.565186 6928 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41774:
name: "default.Table_61514c4f10b54148a57bbaf295205ace"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:08.564543 6926 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41774:
name: "default.Table_c4966a160ae2456ebe5e58c41a9ac129"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:08.550410 6932 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41774:
name: "default.Table_271147f124bf45a88d332b72ac8d3e13"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:08.552663 6931 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41774:
name: "default.Table_49a9f7854b4c4847bce27e16296322b5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:08.553818 6930 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41774:
name: "default.Table_5d5e7f4976094d7a942e0ba75696c8f7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:08.569362 6925 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41774:
name: "default.Table_b6eca8c0167f4d80b6ef4f8125aea68f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:08.567831 6924 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41774:
name: "default.Table_8cc3d0e85b19426f8df95f1894914926"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:42:08.596158 6929 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_472c6a33c87c4f1b9cdb926a2e5a6344 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260807 19:42:08.598152 6930 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_5d5e7f4976094d7a942e0ba75696c8f7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260807 19:42:08.598687 6933 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_93e5970d7fe54e40903e703a8eecd0cf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260807 19:42:08.601374 6931 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_49a9f7854b4c4847bce27e16296322b5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260807 19:42:08.603070 6927 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_ca953b708a5a408898f72c483fc31ecf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260807 19:42:08.604514 6932 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_271147f124bf45a88d332b72ac8d3e13 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260807 19:42:08.605696 6924 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_8cc3d0e85b19426f8df95f1894914926 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260807 19:42:08.606088 6925 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_b6eca8c0167f4d80b6ef4f8125aea68f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260807 19:42:08.607358 6926 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_c4966a160ae2456ebe5e58c41a9ac129 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260807 19:42:08.608942 6928 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_61514c4f10b54148a57bbaf295205ace in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:42:08.615653 6580 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskeMwJpl/build/asan/bin/kudu pid 6896
I20260807 19:42:08.617475 6964 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:08.731809 7597 raft_consensus.cc:482] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1b782e30a8d44e7b94f080aa6327134f)
I20260807 19:42:08.732249 7597 raft_consensus.cc:504] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:08.733341 7597 leader_election.cc:288] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263), 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129)
I20260807 19:42:08.734011 7014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "959ac7807a8b42a59c61a23d4605c054" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7b31edda0c4a48489ad60fc59c44ed31" is_pre_election: true
I20260807 19:42:08.774104 6943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "959ac7807a8b42a59c61a23d4605c054" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1b782e30a8d44e7b94f080aa6327134f" is_pre_election: true
I20260807 19:42:08.774951 6846 leader_election.cc:302] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 959ac7807a8b42a59c61a23d4605c054; no voters: 1b782e30a8d44e7b94f080aa6327134f, 7b31edda0c4a48489ad60fc59c44ed31
I20260807 19:42:08.775712 7597 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:42:08.778335 6580 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskeMwJpl/build/asan/bin/kudu with pid 6896
W20260807 19:42:08.812227 7559 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.6.109.61:39263)
I20260807 19:42:08.812325 7598 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1b782e30a8d44e7b94f080aa6327134f)
I20260807 19:42:08.812583 7598 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:08.813396 7598 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
I20260807 19:42:08.814175 6869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7b31edda0c4a48489ad60fc59c44ed31" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "959ac7807a8b42a59c61a23d4605c054" is_pre_election: true
I20260807 19:42:08.814569 6869 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b31edda0c4a48489ad60fc59c44ed31 in term 4.
I20260807 19:42:08.815240 6990 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31, 959ac7807a8b42a59c61a23d4605c054; no voters:
I20260807 19:42:08.816133 7598 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:42:08.816425 7598 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1b782e30a8d44e7b94f080aa6327134f)
I20260807 19:42:08.816630 7598 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 4 FOLLOWER]: Advancing to term 5
W20260807 19:42:08.817392 7399 heartbeater.cc:644] Failed to heartbeat to 127.6.109.61:39263 (0 consecutive failures): Network error: Failed to send heartbeat to master: recv got EOF from 127.6.109.61:39263 (error 108)
W20260807 19:42:08.818671 6991 connection.cc:535] client connection to 127.6.109.61:39263 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 19:42:08.818957 6991 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 19:42:08.819801 7598 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:08.821090 6869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7b31edda0c4a48489ad60fc59c44ed31" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "959ac7807a8b42a59c61a23d4605c054"
I20260807 19:42:08.821367 6869 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:42:08.824389 6869 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b31edda0c4a48489ad60fc59c44ed31 in term 5.
I20260807 19:42:08.825031 6990 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31, 959ac7807a8b42a59c61a23d4605c054; no voters:
W20260807 19:42:08.830518 7562 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:42:08.814510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.109.61:39263 (local address 127.0.0.1:41788)
0807 19:42:08.814901 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:08.814926 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:08.815102 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:08.828526 (+ 13424us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":106}
I20260807 19:42:08.831573 7598 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 5 election: Requested vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
I20260807 19:42:08.832382 7598 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:42:08.832909 7598 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 5 LEADER]: Becoming Leader. State: Replica: 7b31edda0c4a48489ad60fc59c44ed31, State: Running, Role: LEADER
I20260807 19:42:08.833523 7598 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:08.842955 7601 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b31edda0c4a48489ad60fc59c44ed31. Latest consensus state: current_term: 5 leader_uuid: "7b31edda0c4a48489ad60fc59c44ed31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:08.843493 7601 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: This master's current role is: LEADER
W20260807 19:42:08.846328 7600 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:42:08.821137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.109.61:39263 (local address 127.0.0.1:41800)
0807 19:42:08.844514 (+ 23377us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:08.844770 (+ 256us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":23163,"thread_start_us":23934,"threads_started":1}
W20260807 19:42:08.846828 6991 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection reset by peer (error 104)
I20260807 19:42:08.851554 7603 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:08.853561 7603 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:08.855099 7603 catalog_manager.cc:1259] Loaded cluster ID: a80f7b000fe149f3a73e545a28c78fda
I20260807 19:42:08.855232 7603 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:08.857177 7603 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:08.864822 7603 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: Loaded TSK: 0
W20260807 19:42:08.858760 6991 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:42:08.865435 7603 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:42:08.866653 7603 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260807 19:42:08.867712 7603 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260807 19:42:08.876394 7002 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37396:
name: "default.Table_c4966a160ae2456ebe5e58c41a9ac129"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:08.877350 7001 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37396:
name: "default.Table_ca953b708a5a408898f72c483fc31ecf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:08.877674 7000 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37396:
name: "default.Table_271147f124bf45a88d332b72ac8d3e13"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:08.878206 6999 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37396:
name: "default.Table_93e5970d7fe54e40903e703a8eecd0cf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:08.878401 6998 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37396:
name: "default.Table_49a9f7854b4c4847bce27e16296322b5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:08.879014 6997 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37396:
name: "default.Table_5d5e7f4976094d7a942e0ba75696c8f7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:08.879088 6996 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37396:
name: "default.Table_472c6a33c87c4f1b9cdb926a2e5a6344"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:08.876611 7003 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37396:
name: "default.Table_b6eca8c0167f4d80b6ef4f8125aea68f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:08.880609 7004 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37396:
name: "default.Table_8cc3d0e85b19426f8df95f1894914926"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:08.881692 6995 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37396:
name: "default.Table_61514c4f10b54148a57bbaf295205ace"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:42:08.889895 7002 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_c4966a160ae2456ebe5e58c41a9ac129 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260807 19:42:08.890084 6996 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_472c6a33c87c4f1b9cdb926a2e5a6344 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260807 19:42:08.890131 6999 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_93e5970d7fe54e40903e703a8eecd0cf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260807 19:42:08.890300 6998 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_49a9f7854b4c4847bce27e16296322b5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260807 19:42:08.890475 6997 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_5d5e7f4976094d7a942e0ba75696c8f7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260807 19:42:08.889895 7001 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_ca953b708a5a408898f72c483fc31ecf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260807 19:42:08.891072 7000 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_271147f124bf45a88d332b72ac8d3e13 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260807 19:42:08.956722 7262 heartbeater.cc:644] Failed to heartbeat to 127.6.109.61:39263 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:42:08.996912 7548 heartbeater.cc:503] Master 127.6.109.61:39263 was elected leader, sending a full tablet report...
W20260807 19:42:08.999963 7548 heartbeater.cc:644] Failed to heartbeat to 127.6.109.61:39263 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:42:09.156833 7002 catalog_manager.cc:2148] created HMS catalog entry for table [id=490e9d5bbd5347aaae98c3a81cb3fe1e]
I20260807 19:42:09.167104 6869 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 5 FOLLOWER]: Refusing update from remote peer 7b31edda0c4a48489ad60fc59c44ed31: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
W20260807 19:42:09.167938 6991 consensus_peers.cc:487] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 -> Peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Couldn't send request to peer 1b782e30a8d44e7b94f080aa6327134f. Status: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 19:42:09.168114 7601 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 19:42:09.182214 7597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b31edda0c4a48489ad60fc59c44ed31. Latest consensus state: current_term: 5 leader_uuid: "7b31edda0c4a48489ad60fc59c44ed31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:09.182539 7597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:09.186209 7597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "7b31edda0c4a48489ad60fc59c44ed31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:09.186510 7597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:09.194020 6998 catalog_manager.cc:2148] created HMS catalog entry for table [id=cf4da31cb40447199dc079b8b6e709bc]
I20260807 19:42:09.199950 7598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 5 leader_uuid: "7b31edda0c4a48489ad60fc59c44ed31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:09.200436 7598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:09.213272 6998 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37396:
table { table_name: "default.Table_c4966a160ae2456ebe5e58c41a9ac129" } new_table_name: "default.Table_d6db13f17872439a8858b3d5fd56bfde" modify_external_catalogs: true
I20260807 19:42:09.215332 7002 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37396:
table { table_name: "default.Table_49a9f7854b4c4847bce27e16296322b5" } new_table_name: "default.Table_e60fcc04a04e4a0d9456e55b94757d74" modify_external_catalogs: true
I20260807 19:42:09.260932 7481 tablet_service.cc:1467] Processing CreateTablet for tablet c2f7996c00ea44e8a1b8012ab53e52e2 (DEFAULT_TABLE table=default.table_49a9f7854b4c4847bce27e16296322b5 [id=cf4da31cb40447199dc079b8b6e709bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:09.262717 7481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2f7996c00ea44e8a1b8012ab53e52e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:09.265903 7480 tablet_service.cc:1467] Processing CreateTablet for tablet 1512c8407da64adca2d3019f9b8ed1bd (DEFAULT_TABLE table=default.table_c4966a160ae2456ebe5e58c41a9ac129 [id=490e9d5bbd5347aaae98c3a81cb3fe1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:09.266415 7480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1512c8407da64adca2d3019f9b8ed1bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:09.281414 7620 tablet_bootstrap.cc:492] T 1512c8407da64adca2d3019f9b8ed1bd P bafdff0fd60146539631a476c4257e16: Bootstrap starting.
I20260807 19:42:09.286367 7620 tablet_bootstrap.cc:654] T 1512c8407da64adca2d3019f9b8ed1bd P bafdff0fd60146539631a476c4257e16: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:09.288076 7620 log.cc:824] T 1512c8407da64adca2d3019f9b8ed1bd P bafdff0fd60146539631a476c4257e16: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:09.288884 7332 tablet_service.cc:1467] Processing CreateTablet for tablet c2f7996c00ea44e8a1b8012ab53e52e2 (DEFAULT_TABLE table=default.table_49a9f7854b4c4847bce27e16296322b5 [id=cf4da31cb40447199dc079b8b6e709bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:09.290649 7332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2f7996c00ea44e8a1b8012ab53e52e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:09.297465 7331 tablet_service.cc:1467] Processing CreateTablet for tablet 1512c8407da64adca2d3019f9b8ed1bd (DEFAULT_TABLE table=default.table_c4966a160ae2456ebe5e58c41a9ac129 [id=490e9d5bbd5347aaae98c3a81cb3fe1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:09.298142 7331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1512c8407da64adca2d3019f9b8ed1bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:09.305094 7186 tablet_service.cc:1467] Processing CreateTablet for tablet 1512c8407da64adca2d3019f9b8ed1bd (DEFAULT_TABLE table=default.table_c4966a160ae2456ebe5e58c41a9ac129 [id=490e9d5bbd5347aaae98c3a81cb3fe1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:09.307425 7186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1512c8407da64adca2d3019f9b8ed1bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:09.309446 7187 tablet_service.cc:1467] Processing CreateTablet for tablet c2f7996c00ea44e8a1b8012ab53e52e2 (DEFAULT_TABLE table=default.table_49a9f7854b4c4847bce27e16296322b5 [id=cf4da31cb40447199dc079b8b6e709bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:09.310181 7187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2f7996c00ea44e8a1b8012ab53e52e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:09.314949 7000 catalog_manager.cc:2148] created HMS catalog entry for table [id=a75161def1a34954bf0846ab30a8cc38]
I20260807 19:42:09.325963 7000 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37396:
table { table_name: "default.Table_271147f124bf45a88d332b72ac8d3e13" } new_table_name: "default.Table_830c2e1165884867b9cb12c80f090b48" modify_external_catalogs: true
I20260807 19:42:09.333786 7480 tablet_service.cc:1467] Processing CreateTablet for tablet d328d2e772ed4a6bb8597658f907f0f0 (DEFAULT_TABLE table=default.table_271147f124bf45a88d332b72ac8d3e13 [id=a75161def1a34954bf0846ab30a8cc38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:09.334342 7480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d328d2e772ed4a6bb8597658f907f0f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:09.338558 7185 tablet_service.cc:1467] Processing CreateTablet for tablet d328d2e772ed4a6bb8597658f907f0f0 (DEFAULT_TABLE table=default.table_271147f124bf45a88d332b72ac8d3e13 [id=a75161def1a34954bf0846ab30a8cc38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:09.339165 7185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d328d2e772ed4a6bb8597658f907f0f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:09.348212 7620 tablet_bootstrap.cc:492] T 1512c8407da64adca2d3019f9b8ed1bd P bafdff0fd60146539631a476c4257e16: No bootstrap required, opened a new log
I20260807 19:42:09.349001 7622 tablet_bootstrap.cc:492] T 1512c8407da64adca2d3019f9b8ed1bd P 35b5ecc2c9ef47c2bbe35653633e14da: Bootstrap starting.
I20260807 19:42:09.349875 7620 ts_tablet_manager.cc:1397] T 1512c8407da64adca2d3019f9b8ed1bd P bafdff0fd60146539631a476c4257e16: Time spent bootstrapping tablet: real 0.069s user 0.018s sys 0.015s
I20260807 19:42:09.351486 7331 tablet_service.cc:1467] Processing CreateTablet for tablet d328d2e772ed4a6bb8597658f907f0f0 (DEFAULT_TABLE table=default.table_271147f124bf45a88d332b72ac8d3e13 [id=a75161def1a34954bf0846ab30a8cc38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:09.352274 7331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d328d2e772ed4a6bb8597658f907f0f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:09.352783 7624 tablet_bootstrap.cc:492] T 1512c8407da64adca2d3019f9b8ed1bd P 051d464d644a4d4486462dfb18274c6a: Bootstrap starting.
I20260807 19:42:09.353294 7622 tablet_bootstrap.cc:654] T 1512c8407da64adca2d3019f9b8ed1bd P 35b5ecc2c9ef47c2bbe35653633e14da: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:09.354851 7622 log.cc:824] T 1512c8407da64adca2d3019f9b8ed1bd P 35b5ecc2c9ef47c2bbe35653633e14da: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:09.357986 7624 tablet_bootstrap.cc:654] T 1512c8407da64adca2d3019f9b8ed1bd P 051d464d644a4d4486462dfb18274c6a: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:09.360106 7624 log.cc:824] T 1512c8407da64adca2d3019f9b8ed1bd P 051d464d644a4d4486462dfb18274c6a: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:09.367961 7620 raft_consensus.cc:348] T 1512c8407da64adca2d3019f9b8ed1bd P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:09.368552 7620 raft_consensus.cc:374] T 1512c8407da64adca2d3019f9b8ed1bd P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:09.368674 7620 raft_consensus.cc:729] T 1512c8407da64adca2d3019f9b8ed1bd P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bafdff0fd60146539631a476c4257e16, State: Initialized, Role: FOLLOWER
I20260807 19:42:09.369436 7620 consensus_queue.cc:260] T 1512c8407da64adca2d3019f9b8ed1bd P bafdff0fd60146539631a476c4257e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:09.370172 7624 tablet_bootstrap.cc:492] T 1512c8407da64adca2d3019f9b8ed1bd P 051d464d644a4d4486462dfb18274c6a: No bootstrap required, opened a new log
I20260807 19:42:09.370216 7622 tablet_bootstrap.cc:492] T 1512c8407da64adca2d3019f9b8ed1bd P 35b5ecc2c9ef47c2bbe35653633e14da: No bootstrap required, opened a new log
I20260807 19:42:09.370572 7624 ts_tablet_manager.cc:1397] T 1512c8407da64adca2d3019f9b8ed1bd P 051d464d644a4d4486462dfb18274c6a: Time spent bootstrapping tablet: real 0.018s user 0.010s sys 0.005s
I20260807 19:42:09.370585 7622 ts_tablet_manager.cc:1397] T 1512c8407da64adca2d3019f9b8ed1bd P 35b5ecc2c9ef47c2bbe35653633e14da: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.007s
I20260807 19:42:09.378474 7620 ts_tablet_manager.cc:1428] T 1512c8407da64adca2d3019f9b8ed1bd P bafdff0fd60146539631a476c4257e16: Time spent starting tablet: real 0.026s user 0.021s sys 0.001s
I20260807 19:42:09.378877 7546 heartbeater.cc:503] Master 127.6.109.60:43129 was elected leader, sending a full tablet report...
I20260807 19:42:09.381657 7620 tablet_bootstrap.cc:492] T c2f7996c00ea44e8a1b8012ab53e52e2 P bafdff0fd60146539631a476c4257e16: Bootstrap starting.
I20260807 19:42:09.384166 7620 tablet_bootstrap.cc:654] T c2f7996c00ea44e8a1b8012ab53e52e2 P bafdff0fd60146539631a476c4257e16: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:09.383904 7624 raft_consensus.cc:348] T 1512c8407da64adca2d3019f9b8ed1bd P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:09.384553 7624 raft_consensus.cc:374] T 1512c8407da64adca2d3019f9b8ed1bd P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:09.384708 7624 raft_consensus.cc:729] T 1512c8407da64adca2d3019f9b8ed1bd P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 051d464d644a4d4486462dfb18274c6a, State: Initialized, Role: FOLLOWER
I20260807 19:42:09.384963 7622 raft_consensus.cc:348] T 1512c8407da64adca2d3019f9b8ed1bd P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:09.385445 7622 raft_consensus.cc:374] T 1512c8407da64adca2d3019f9b8ed1bd P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:09.385558 7622 raft_consensus.cc:729] T 1512c8407da64adca2d3019f9b8ed1bd P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35b5ecc2c9ef47c2bbe35653633e14da, State: Initialized, Role: FOLLOWER
I20260807 19:42:09.385649 7624 consensus_queue.cc:260] T 1512c8407da64adca2d3019f9b8ed1bd P 051d464d644a4d4486462dfb18274c6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:09.386353 7622 consensus_queue.cc:260] T 1512c8407da64adca2d3019f9b8ed1bd P 35b5ecc2c9ef47c2bbe35653633e14da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:09.390769 7620 tablet_bootstrap.cc:492] T c2f7996c00ea44e8a1b8012ab53e52e2 P bafdff0fd60146539631a476c4257e16: No bootstrap required, opened a new log
I20260807 19:42:09.390995 7620 ts_tablet_manager.cc:1397] T c2f7996c00ea44e8a1b8012ab53e52e2 P bafdff0fd60146539631a476c4257e16: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.004s
I20260807 19:42:09.392462 7620 raft_consensus.cc:348] T c2f7996c00ea44e8a1b8012ab53e52e2 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } }
I20260807 19:42:09.392803 7620 raft_consensus.cc:374] T c2f7996c00ea44e8a1b8012ab53e52e2 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:09.392911 7620 raft_consensus.cc:729] T c2f7996c00ea44e8a1b8012ab53e52e2 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bafdff0fd60146539631a476c4257e16, State: Initialized, Role: FOLLOWER
I20260807 19:42:09.393234 7620 consensus_queue.cc:260] T c2f7996c00ea44e8a1b8012ab53e52e2 P bafdff0fd60146539631a476c4257e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } }
I20260807 19:42:09.394924 7622 ts_tablet_manager.cc:1428] T 1512c8407da64adca2d3019f9b8ed1bd P 35b5ecc2c9ef47c2bbe35653633e14da: Time spent starting tablet: real 0.024s user 0.021s sys 0.001s
I20260807 19:42:09.395249 7397 heartbeater.cc:503] Master 127.6.109.60:43129 was elected leader, sending a full tablet report...
I20260807 19:42:09.396064 7620 ts_tablet_manager.cc:1428] T c2f7996c00ea44e8a1b8012ab53e52e2 P bafdff0fd60146539631a476c4257e16: Time spent starting tablet: real 0.005s user 0.002s sys 0.001s
I20260807 19:42:09.398250 7620 tablet_bootstrap.cc:492] T d328d2e772ed4a6bb8597658f907f0f0 P bafdff0fd60146539631a476c4257e16: Bootstrap starting.
I20260807 19:42:09.400008 7622 tablet_bootstrap.cc:492] T c2f7996c00ea44e8a1b8012ab53e52e2 P 35b5ecc2c9ef47c2bbe35653633e14da: Bootstrap starting.
I20260807 19:42:09.400568 7620 tablet_bootstrap.cc:654] T d328d2e772ed4a6bb8597658f907f0f0 P bafdff0fd60146539631a476c4257e16: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:09.401839 7622 tablet_bootstrap.cc:654] T c2f7996c00ea44e8a1b8012ab53e52e2 P 35b5ecc2c9ef47c2bbe35653633e14da: Neither blocks nor log segments found. Creating new log.
W20260807 19:42:09.404570 7400 tablet.cc:2369] T 1512c8407da64adca2d3019f9b8ed1bd P 35b5ecc2c9ef47c2bbe35653633e14da: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:42:09.411540 7622 tablet_bootstrap.cc:492] T c2f7996c00ea44e8a1b8012ab53e52e2 P 35b5ecc2c9ef47c2bbe35653633e14da: No bootstrap required, opened a new log
I20260807 19:42:09.411768 7622 ts_tablet_manager.cc:1397] T c2f7996c00ea44e8a1b8012ab53e52e2 P 35b5ecc2c9ef47c2bbe35653633e14da: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.006s
I20260807 19:42:09.413151 7622 raft_consensus.cc:348] T c2f7996c00ea44e8a1b8012ab53e52e2 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } }
I20260807 19:42:09.413475 7622 raft_consensus.cc:374] T c2f7996c00ea44e8a1b8012ab53e52e2 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:09.413597 7622 raft_consensus.cc:729] T c2f7996c00ea44e8a1b8012ab53e52e2 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35b5ecc2c9ef47c2bbe35653633e14da, State: Initialized, Role: FOLLOWER
I20260807 19:42:09.414018 7622 consensus_queue.cc:260] T c2f7996c00ea44e8a1b8012ab53e52e2 P 35b5ecc2c9ef47c2bbe35653633e14da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } }
I20260807 19:42:09.419173 7622 ts_tablet_manager.cc:1428] T c2f7996c00ea44e8a1b8012ab53e52e2 P 35b5ecc2c9ef47c2bbe35653633e14da: Time spent starting tablet: real 0.007s user 0.000s sys 0.004s
I20260807 19:42:09.423481 7622 tablet_bootstrap.cc:492] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da: Bootstrap starting.
I20260807 19:42:09.425997 7622 tablet_bootstrap.cc:654] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:09.426553 7255 heartbeater.cc:503] Master 127.6.109.60:43129 was elected leader, sending a full tablet report...
I20260807 19:42:09.427634 7624 ts_tablet_manager.cc:1428] T 1512c8407da64adca2d3019f9b8ed1bd P 051d464d644a4d4486462dfb18274c6a: Time spent starting tablet: real 0.057s user 0.033s sys 0.020s
I20260807 19:42:09.428612 7620 tablet_bootstrap.cc:492] T d328d2e772ed4a6bb8597658f907f0f0 P bafdff0fd60146539631a476c4257e16: No bootstrap required, opened a new log
I20260807 19:42:09.428712 7622 tablet_bootstrap.cc:492] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da: No bootstrap required, opened a new log
I20260807 19:42:09.428812 7620 ts_tablet_manager.cc:1397] T d328d2e772ed4a6bb8597658f907f0f0 P bafdff0fd60146539631a476c4257e16: Time spent bootstrapping tablet: real 0.031s user 0.015s sys 0.012s
I20260807 19:42:09.428862 7622 ts_tablet_manager.cc:1397] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260807 19:42:09.429745 7620 raft_consensus.cc:348] T d328d2e772ed4a6bb8597658f907f0f0 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:09.429744 7622 raft_consensus.cc:348] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:09.430028 7620 raft_consensus.cc:374] T d328d2e772ed4a6bb8597658f907f0f0 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:09.430054 7622 raft_consensus.cc:374] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:09.430112 7620 raft_consensus.cc:729] T d328d2e772ed4a6bb8597658f907f0f0 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bafdff0fd60146539631a476c4257e16, State: Initialized, Role: FOLLOWER
I20260807 19:42:09.430130 7622 raft_consensus.cc:729] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35b5ecc2c9ef47c2bbe35653633e14da, State: Initialized, Role: FOLLOWER
I20260807 19:42:09.430387 7620 consensus_queue.cc:260] T d328d2e772ed4a6bb8597658f907f0f0 P bafdff0fd60146539631a476c4257e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:09.431077 7620 ts_tablet_manager.cc:1428] T d328d2e772ed4a6bb8597658f907f0f0 P bafdff0fd60146539631a476c4257e16: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 19:42:09.430505 7622 consensus_queue.cc:260] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:09.433022 7624 tablet_bootstrap.cc:492] T d328d2e772ed4a6bb8597658f907f0f0 P 051d464d644a4d4486462dfb18274c6a: Bootstrap starting.
I20260807 19:42:09.433714 7622 ts_tablet_manager.cc:1428] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da: Time spent starting tablet: real 0.005s user 0.002s sys 0.001s
I20260807 19:42:09.435568 7624 tablet_bootstrap.cc:654] T d328d2e772ed4a6bb8597658f907f0f0 P 051d464d644a4d4486462dfb18274c6a: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:09.438025 7624 tablet_bootstrap.cc:492] T d328d2e772ed4a6bb8597658f907f0f0 P 051d464d644a4d4486462dfb18274c6a: No bootstrap required, opened a new log
I20260807 19:42:09.438195 7624 ts_tablet_manager.cc:1397] T d328d2e772ed4a6bb8597658f907f0f0 P 051d464d644a4d4486462dfb18274c6a: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 19:42:09.439316 7624 raft_consensus.cc:348] T d328d2e772ed4a6bb8597658f907f0f0 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:09.439565 7624 raft_consensus.cc:374] T d328d2e772ed4a6bb8597658f907f0f0 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:09.439667 7624 raft_consensus.cc:729] T d328d2e772ed4a6bb8597658f907f0f0 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 051d464d644a4d4486462dfb18274c6a, State: Initialized, Role: FOLLOWER
I20260807 19:42:09.439962 7624 consensus_queue.cc:260] T d328d2e772ed4a6bb8597658f907f0f0 P 051d464d644a4d4486462dfb18274c6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:09.440543 7624 ts_tablet_manager.cc:1428] T d328d2e772ed4a6bb8597658f907f0f0 P 051d464d644a4d4486462dfb18274c6a: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260807 19:42:09.442621 7624 tablet_bootstrap.cc:492] T c2f7996c00ea44e8a1b8012ab53e52e2 P 051d464d644a4d4486462dfb18274c6a: Bootstrap starting.
I20260807 19:42:09.444692 7624 tablet_bootstrap.cc:654] T c2f7996c00ea44e8a1b8012ab53e52e2 P 051d464d644a4d4486462dfb18274c6a: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:09.448681 7624 tablet_bootstrap.cc:492] T c2f7996c00ea44e8a1b8012ab53e52e2 P 051d464d644a4d4486462dfb18274c6a: No bootstrap required, opened a new log
I20260807 19:42:09.448879 7624 ts_tablet_manager.cc:1397] T c2f7996c00ea44e8a1b8012ab53e52e2 P 051d464d644a4d4486462dfb18274c6a: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260807 19:42:09.449822 7624 raft_consensus.cc:348] T c2f7996c00ea44e8a1b8012ab53e52e2 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } }
I20260807 19:42:09.450080 7624 raft_consensus.cc:374] T c2f7996c00ea44e8a1b8012ab53e52e2 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:09.450282 7624 raft_consensus.cc:729] T c2f7996c00ea44e8a1b8012ab53e52e2 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 051d464d644a4d4486462dfb18274c6a, State: Initialized, Role: FOLLOWER
I20260807 19:42:09.450681 7624 consensus_queue.cc:260] T c2f7996c00ea44e8a1b8012ab53e52e2 P 051d464d644a4d4486462dfb18274c6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } }
I20260807 19:42:09.451423 7624 ts_tablet_manager.cc:1428] T c2f7996c00ea44e8a1b8012ab53e52e2 P 051d464d644a4d4486462dfb18274c6a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 19:42:09.455575 6996 catalog_manager.cc:2148] created HMS catalog entry for table [id=0b2f82ef918b47d59b34a5b5757b9486]
W20260807 19:42:09.465013 7549 tablet.cc:2369] T 1512c8407da64adca2d3019f9b8ed1bd P bafdff0fd60146539631a476c4257e16: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:42:09.466571 6580 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskeMwJpl/build/asan/bin/kudu
/tmp/dist-test-taskeMwJpl/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.109.61:39263
--webserver_interface=127.6.109.61
--webserver_port=37257
--builtin_ntp_servers=127.6.109.20:40195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129
--hive_metastore_uris=thrift://127.0.0.1:38987
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260807 19:42:09.476425 7629 raft_consensus.cc:482] T d328d2e772ed4a6bb8597658f907f0f0 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:09.476783 7629 raft_consensus.cc:504] T d328d2e772ed4a6bb8597658f907f0f0 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:09.477479 7186 tablet_service.cc:1467] Processing CreateTablet for tablet 248c1670db7244cfac99dd4817b513a4 (DEFAULT_TABLE table=default.table_472c6a33c87c4f1b9cdb926a2e5a6344 [id=0b2f82ef918b47d59b34a5b5757b9486]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:09.478137 7186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 248c1670db7244cfac99dd4817b513a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:09.478200 7331 tablet_service.cc:1467] Processing CreateTablet for tablet 248c1670db7244cfac99dd4817b513a4 (DEFAULT_TABLE table=default.table_472c6a33c87c4f1b9cdb926a2e5a6344 [id=0b2f82ef918b47d59b34a5b5757b9486]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:09.478655 7331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 248c1670db7244cfac99dd4817b513a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:09.479142 7629 leader_election.cc:288] T d328d2e772ed4a6bb8597658f907f0f0 P 051d464d644a4d4486462dfb18274c6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
I20260807 19:42:09.479715 7480 tablet_service.cc:1467] Processing CreateTablet for tablet 248c1670db7244cfac99dd4817b513a4 (DEFAULT_TABLE table=default.table_472c6a33c87c4f1b9cdb926a2e5a6344 [id=0b2f82ef918b47d59b34a5b5757b9486]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:09.480154 7480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 248c1670db7244cfac99dd4817b513a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:09.484422 7624 tablet_bootstrap.cc:492] T 248c1670db7244cfac99dd4817b513a4 P 051d464d644a4d4486462dfb18274c6a: Bootstrap starting.
I20260807 19:42:09.486171 7624 tablet_bootstrap.cc:654] T 248c1670db7244cfac99dd4817b513a4 P 051d464d644a4d4486462dfb18274c6a: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:09.494035 7622 tablet_bootstrap.cc:492] T 248c1670db7244cfac99dd4817b513a4 P 35b5ecc2c9ef47c2bbe35653633e14da: Bootstrap starting.
I20260807 19:42:09.496286 7622 tablet_bootstrap.cc:654] T 248c1670db7244cfac99dd4817b513a4 P 35b5ecc2c9ef47c2bbe35653633e14da: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:09.498274 7620 tablet_bootstrap.cc:492] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16: Bootstrap starting.
I20260807 19:42:09.500710 7620 tablet_bootstrap.cc:654] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:09.502175 6996 master_service.cc:946] ReplaceTablet: received request to replace tablet 1512c8407da64adca2d3019f9b8ed1bd from {username='slave'} at 127.0.0.1:37396
I20260807 19:42:09.515865 7624 tablet_bootstrap.cc:492] T 248c1670db7244cfac99dd4817b513a4 P 051d464d644a4d4486462dfb18274c6a: No bootstrap required, opened a new log
I20260807 19:42:09.516067 7624 ts_tablet_manager.cc:1397] T 248c1670db7244cfac99dd4817b513a4 P 051d464d644a4d4486462dfb18274c6a: Time spent bootstrapping tablet: real 0.032s user 0.009s sys 0.017s
I20260807 19:42:09.516953 7624 raft_consensus.cc:348] T 248c1670db7244cfac99dd4817b513a4 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:09.517236 7624 raft_consensus.cc:374] T 248c1670db7244cfac99dd4817b513a4 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:09.517350 7624 raft_consensus.cc:729] T 248c1670db7244cfac99dd4817b513a4 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 051d464d644a4d4486462dfb18274c6a, State: Initialized, Role: FOLLOWER
I20260807 19:42:09.517608 7624 consensus_queue.cc:260] T 248c1670db7244cfac99dd4817b513a4 P 051d464d644a4d4486462dfb18274c6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:09.518225 7624 ts_tablet_manager.cc:1428] T 248c1670db7244cfac99dd4817b513a4 P 051d464d644a4d4486462dfb18274c6a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 19:42:09.523243 7622 tablet_bootstrap.cc:492] T 248c1670db7244cfac99dd4817b513a4 P 35b5ecc2c9ef47c2bbe35653633e14da: No bootstrap required, opened a new log
I20260807 19:42:09.523469 7622 ts_tablet_manager.cc:1397] T 248c1670db7244cfac99dd4817b513a4 P 35b5ecc2c9ef47c2bbe35653633e14da: Time spent bootstrapping tablet: real 0.030s user 0.007s sys 0.008s
I20260807 19:42:09.523828 6996 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: Sending DeleteTablet for 3 replicas of tablet 1512c8407da64adca2d3019f9b8ed1bd
I20260807 19:42:09.524235 7501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d328d2e772ed4a6bb8597658f907f0f0" candidate_uuid: "051d464d644a4d4486462dfb18274c6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bafdff0fd60146539631a476c4257e16" is_pre_election: true
I20260807 19:42:09.524356 7622 raft_consensus.cc:348] T 248c1670db7244cfac99dd4817b513a4 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:09.524631 6996 catalog_manager.cc:6565] ReplaceTablet: tablet 1512c8407da64adca2d3019f9b8ed1bd deleted and replaced by tablet 0d47a7a1285948468c0e6e1bddcd26d4
I20260807 19:42:09.524917 7501 raft_consensus.cc:2454] T d328d2e772ed4a6bb8597658f907f0f0 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 051d464d644a4d4486462dfb18274c6a in term 0.
I20260807 19:42:09.525368 7186 tablet_service.cc:1514] Processing DeleteTablet for tablet 1512c8407da64adca2d3019f9b8ed1bd with delete_type TABLET_DATA_DELETED (replaced by tablet 0d47a7a1285948468c0e6e1bddcd26d4) from {username='slave'} at 127.0.0.1:51700
I20260807 19:42:09.526376 7480 tablet_service.cc:1514] Processing DeleteTablet for tablet 1512c8407da64adca2d3019f9b8ed1bd with delete_type TABLET_DATA_DELETED (replaced by tablet 0d47a7a1285948468c0e6e1bddcd26d4) from {username='slave'} at 127.0.0.1:57214
I20260807 19:42:09.528297 7620 tablet_bootstrap.cc:492] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16: No bootstrap required, opened a new log
I20260807 19:42:09.528447 7620 ts_tablet_manager.cc:1397] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16: Time spent bootstrapping tablet: real 0.030s user 0.010s sys 0.001s
I20260807 19:42:09.530004 7620 raft_consensus.cc:348] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:09.530267 7620 raft_consensus.cc:374] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:09.530359 7620 raft_consensus.cc:729] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bafdff0fd60146539631a476c4257e16, State: Initialized, Role: FOLLOWER
I20260807 19:42:09.532580 7620 consensus_queue.cc:260] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:09.533270 7620 ts_tablet_manager.cc:1428] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260807 19:42:09.533836 7140 leader_election.cc:302] T d328d2e772ed4a6bb8597658f907f0f0 P 051d464d644a4d4486462dfb18274c6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051d464d644a4d4486462dfb18274c6a, bafdff0fd60146539631a476c4257e16; no voters:
I20260807 19:42:09.534796 7629 raft_consensus.cc:2781] T d328d2e772ed4a6bb8597658f907f0f0 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:42:09.534963 7629 raft_consensus.cc:482] T d328d2e772ed4a6bb8597658f907f0f0 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:42:09.535051 7629 raft_consensus.cc:3037] T d328d2e772ed4a6bb8597658f907f0f0 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:09.526793 7331 tablet_service.cc:1514] Processing DeleteTablet for tablet 1512c8407da64adca2d3019f9b8ed1bd with delete_type TABLET_DATA_DELETED (replaced by tablet 0d47a7a1285948468c0e6e1bddcd26d4) from {username='slave'} at 127.0.0.1:60860
I20260807 19:42:09.538676 7629 raft_consensus.cc:504] T d328d2e772ed4a6bb8597658f907f0f0 P 051d464d644a4d4486462dfb18274c6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:09.539623 7629 leader_election.cc:288] T d328d2e772ed4a6bb8597658f907f0f0 P 051d464d644a4d4486462dfb18274c6a [CANDIDATE]: Term 1 election: Requested vote from peers 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
I20260807 19:42:09.532320 7352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d328d2e772ed4a6bb8597658f907f0f0" candidate_uuid: "051d464d644a4d4486462dfb18274c6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" is_pre_election: true
I20260807 19:42:09.540758 7352 raft_consensus.cc:2454] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 051d464d644a4d4486462dfb18274c6a in term 0.
I20260807 19:42:09.541718 7622 raft_consensus.cc:374] T 248c1670db7244cfac99dd4817b513a4 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:09.541887 7622 raft_consensus.cc:729] T 248c1670db7244cfac99dd4817b513a4 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35b5ecc2c9ef47c2bbe35653633e14da, State: Initialized, Role: FOLLOWER
I20260807 19:42:09.542179 7501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d328d2e772ed4a6bb8597658f907f0f0" candidate_uuid: "051d464d644a4d4486462dfb18274c6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bafdff0fd60146539631a476c4257e16"
I20260807 19:42:09.542173 7622 consensus_queue.cc:260] T 248c1670db7244cfac99dd4817b513a4 P 35b5ecc2c9ef47c2bbe35653633e14da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:09.542320 7352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d328d2e772ed4a6bb8597658f907f0f0" candidate_uuid: "051d464d644a4d4486462dfb18274c6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35b5ecc2c9ef47c2bbe35653633e14da"
I20260807 19:42:09.542405 7501 raft_consensus.cc:3037] T d328d2e772ed4a6bb8597658f907f0f0 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:09.542580 7352 raft_consensus.cc:3037] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:09.547477 7501 raft_consensus.cc:2454] T d328d2e772ed4a6bb8597658f907f0f0 P bafdff0fd60146539631a476c4257e16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 051d464d644a4d4486462dfb18274c6a in term 1.
I20260807 19:42:09.547567 7622 ts_tablet_manager.cc:1428] T 248c1670db7244cfac99dd4817b513a4 P 35b5ecc2c9ef47c2bbe35653633e14da: Time spent starting tablet: real 0.024s user 0.002s sys 0.002s
I20260807 19:42:09.548418 7642 tablet_replica.cc:330] T 1512c8407da64adca2d3019f9b8ed1bd P bafdff0fd60146539631a476c4257e16: stopping tablet replica
I20260807 19:42:09.548983 7642 raft_consensus.cc:2229] T 1512c8407da64adca2d3019f9b8ed1bd P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:09.549499 7642 raft_consensus.cc:2258] T 1512c8407da64adca2d3019f9b8ed1bd P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:09.551906 7642 ts_tablet_manager.cc:1905] T 1512c8407da64adca2d3019f9b8ed1bd P bafdff0fd60146539631a476c4257e16: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:09.554786 7332 tablet_service.cc:1467] Processing CreateTablet for tablet 0d47a7a1285948468c0e6e1bddcd26d4 (DEFAULT_TABLE table=default.table_c4966a160ae2456ebe5e58c41a9ac129 [id=490e9d5bbd5347aaae98c3a81cb3fe1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:09.554797 7187 tablet_service.cc:1467] Processing CreateTablet for tablet 0d47a7a1285948468c0e6e1bddcd26d4 (DEFAULT_TABLE table=default.table_c4966a160ae2456ebe5e58c41a9ac129 [id=490e9d5bbd5347aaae98c3a81cb3fe1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:09.555260 7332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d47a7a1285948468c0e6e1bddcd26d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:09.555325 7187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d47a7a1285948468c0e6e1bddcd26d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:09.557449 6996 master_service.cc:946] ReplaceTablet: received request to replace tablet 1512c8407da64adca2d3019f9b8ed1bd from {username='slave'} at 127.0.0.1:37396
I20260807 19:42:09.557864 7480 tablet_service.cc:1467] Processing CreateTablet for tablet 0d47a7a1285948468c0e6e1bddcd26d4 (DEFAULT_TABLE table=default.table_c4966a160ae2456ebe5e58c41a9ac129 [id=490e9d5bbd5347aaae98c3a81cb3fe1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:09.558024 6996 master_service.cc:946] ReplaceTablet: received request to replace tablet c2f7996c00ea44e8a1b8012ab53e52e2 from {username='slave'} at 127.0.0.1:37396
I20260807 19:42:09.558336 7480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d47a7a1285948468c0e6e1bddcd26d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:09.564733 7641 tablet_replica.cc:330] T 1512c8407da64adca2d3019f9b8ed1bd P 051d464d644a4d4486462dfb18274c6a: stopping tablet replica
I20260807 19:42:09.565290 7641 raft_consensus.cc:2229] T 1512c8407da64adca2d3019f9b8ed1bd P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:09.565810 7641 raft_consensus.cc:2258] T 1512c8407da64adca2d3019f9b8ed1bd P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:09.566614 7352 raft_consensus.cc:2454] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 051d464d644a4d4486462dfb18274c6a in term 1.
I20260807 19:42:09.566898 7624 tablet_bootstrap.cc:492] T 0d47a7a1285948468c0e6e1bddcd26d4 P 051d464d644a4d4486462dfb18274c6a: Bootstrap starting.
I20260807 19:42:09.568775 7624 tablet_bootstrap.cc:654] T 0d47a7a1285948468c0e6e1bddcd26d4 P 051d464d644a4d4486462dfb18274c6a: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:09.571419 6996 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: Sending DeleteTablet for 3 replicas of tablet c2f7996c00ea44e8a1b8012ab53e52e2
I20260807 19:42:09.571667 7620 tablet_bootstrap.cc:492] T 0d47a7a1285948468c0e6e1bddcd26d4 P bafdff0fd60146539631a476c4257e16: Bootstrap starting.
I20260807 19:42:09.573398 7620 tablet_bootstrap.cc:654] T 0d47a7a1285948468c0e6e1bddcd26d4 P bafdff0fd60146539631a476c4257e16: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:09.575186 6996 catalog_manager.cc:6565] ReplaceTablet: tablet c2f7996c00ea44e8a1b8012ab53e52e2 deleted and replaced by tablet 9a45d0765a2c4d93ac9448496ce9c325
I20260807 19:42:09.575465 7481 tablet_service.cc:1514] Processing DeleteTablet for tablet c2f7996c00ea44e8a1b8012ab53e52e2 with delete_type TABLET_DATA_DELETED (replaced by tablet 9a45d0765a2c4d93ac9448496ce9c325) from {username='slave'} at 127.0.0.1:57214
I20260807 19:42:09.576538 7187 tablet_service.cc:1514] Processing DeleteTablet for tablet c2f7996c00ea44e8a1b8012ab53e52e2 with delete_type TABLET_DATA_DELETED (replaced by tablet 9a45d0765a2c4d93ac9448496ce9c325) from {username='slave'} at 127.0.0.1:51700
I20260807 19:42:09.575517 6996 master_service.cc:946] ReplaceTablet: received request to replace tablet c2f7996c00ea44e8a1b8012ab53e52e2 from {username='slave'} at 127.0.0.1:37396
I20260807 19:42:09.577498 7331 tablet_service.cc:1514] Processing DeleteTablet for tablet c2f7996c00ea44e8a1b8012ab53e52e2 with delete_type TABLET_DATA_DELETED (replaced by tablet 9a45d0765a2c4d93ac9448496ce9c325) from {username='slave'} at 127.0.0.1:60860
I20260807 19:42:09.581333 7140 leader_election.cc:302] T d328d2e772ed4a6bb8597658f907f0f0 P 051d464d644a4d4486462dfb18274c6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051d464d644a4d4486462dfb18274c6a, bafdff0fd60146539631a476c4257e16; no voters:
I20260807 19:42:09.581949 7629 raft_consensus.cc:2781] T d328d2e772ed4a6bb8597658f907f0f0 P 051d464d644a4d4486462dfb18274c6a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:09.583689 7620 tablet_bootstrap.cc:492] T 0d47a7a1285948468c0e6e1bddcd26d4 P bafdff0fd60146539631a476c4257e16: No bootstrap required, opened a new log
I20260807 19:42:09.583894 7620 ts_tablet_manager.cc:1397] T 0d47a7a1285948468c0e6e1bddcd26d4 P bafdff0fd60146539631a476c4257e16: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.008s
I20260807 19:42:09.585260 7187 tablet_service.cc:1467] Processing CreateTablet for tablet 9a45d0765a2c4d93ac9448496ce9c325 (DEFAULT_TABLE table=default.table_49a9f7854b4c4847bce27e16296322b5 [id=cf4da31cb40447199dc079b8b6e709bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:09.566709 7643 tablet_replica.cc:330] T 1512c8407da64adca2d3019f9b8ed1bd P 35b5ecc2c9ef47c2bbe35653633e14da: stopping tablet replica
I20260807 19:42:09.567657 7622 tablet_bootstrap.cc:492] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da: Bootstrap starting.
I20260807 19:42:09.585856 7187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a45d0765a2c4d93ac9448496ce9c325. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:09.586689 7480 tablet_service.cc:1467] Processing CreateTablet for tablet 9a45d0765a2c4d93ac9448496ce9c325 (DEFAULT_TABLE table=default.table_49a9f7854b4c4847bce27e16296322b5 [id=cf4da31cb40447199dc079b8b6e709bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:09.587172 7480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a45d0765a2c4d93ac9448496ce9c325. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:09.588346 7622 tablet_bootstrap.cc:654] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:09.589202 7331 tablet_service.cc:1467] Processing CreateTablet for tablet 9a45d0765a2c4d93ac9448496ce9c325 (DEFAULT_TABLE table=default.table_49a9f7854b4c4847bce27e16296322b5 [id=cf4da31cb40447199dc079b8b6e709bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:09.589681 7331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a45d0765a2c4d93ac9448496ce9c325. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:09.589766 7624 tablet_bootstrap.cc:492] T 0d47a7a1285948468c0e6e1bddcd26d4 P 051d464d644a4d4486462dfb18274c6a: No bootstrap required, opened a new log
I20260807 19:42:09.589970 7624 ts_tablet_manager.cc:1397] T 0d47a7a1285948468c0e6e1bddcd26d4 P 051d464d644a4d4486462dfb18274c6a: Time spent bootstrapping tablet: real 0.023s user 0.008s sys 0.002s
I20260807 19:42:09.589993 7643 raft_consensus.cc:2229] T 1512c8407da64adca2d3019f9b8ed1bd P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:09.590632 7643 raft_consensus.cc:2258] T 1512c8407da64adca2d3019f9b8ed1bd P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:09.590664 6996 master_service.cc:946] ReplaceTablet: received request to replace tablet 1512c8407da64adca2d3019f9b8ed1bd from {username='slave'} at 127.0.0.1:37396
I20260807 19:42:09.590961 7624 raft_consensus.cc:348] T 0d47a7a1285948468c0e6e1bddcd26d4 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:09.591461 7624 raft_consensus.cc:374] T 0d47a7a1285948468c0e6e1bddcd26d4 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:09.591549 7624 raft_consensus.cc:729] T 0d47a7a1285948468c0e6e1bddcd26d4 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 051d464d644a4d4486462dfb18274c6a, State: Initialized, Role: FOLLOWER
I20260807 19:42:09.591843 7624 consensus_queue.cc:260] T 0d47a7a1285948468c0e6e1bddcd26d4 P 051d464d644a4d4486462dfb18274c6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:09.592211 6996 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37396:
name: "default.Table_1455c4f504ff4603bc11da85f16cc69f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:09.594259 7643 ts_tablet_manager.cc:1905] T 1512c8407da64adca2d3019f9b8ed1bd P 35b5ecc2c9ef47c2bbe35653633e14da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:09.598187 7642 ts_tablet_manager.cc:1918] T 1512c8407da64adca2d3019f9b8ed1bd P bafdff0fd60146539631a476c4257e16: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260807 19:42:09.598476 7642 log.cc:1196] T 1512c8407da64adca2d3019f9b8ed1bd P bafdff0fd60146539631a476c4257e16: Deleting WAL directory at /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-2/wal/wals/1512c8407da64adca2d3019f9b8ed1bd
I20260807 19:42:09.599062 7642 ts_tablet_manager.cc:1939] T 1512c8407da64adca2d3019f9b8ed1bd P bafdff0fd60146539631a476c4257e16: Deleting consensus metadata
I20260807 19:42:09.599653 7627 raft_consensus.cc:482] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:09.599870 7627 raft_consensus.cc:504] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:09.600961 7641 ts_tablet_manager.cc:1905] T 1512c8407da64adca2d3019f9b8ed1bd P 051d464d644a4d4486462dfb18274c6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:09.601348 7643 ts_tablet_manager.cc:1918] T 1512c8407da64adca2d3019f9b8ed1bd P 35b5ecc2c9ef47c2bbe35653633e14da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260807 19:42:09.601533 7643 log.cc:1196] T 1512c8407da64adca2d3019f9b8ed1bd P 35b5ecc2c9ef47c2bbe35653633e14da: Deleting WAL directory at /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-1/wal/wals/1512c8407da64adca2d3019f9b8ed1bd
I20260807 19:42:09.602012 7627 leader_election.cc:288] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467), 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483)
I20260807 19:42:09.584702 7620 raft_consensus.cc:348] T 0d47a7a1285948468c0e6e1bddcd26d4 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:09.602761 7620 raft_consensus.cc:374] T 0d47a7a1285948468c0e6e1bddcd26d4 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:09.602871 7620 raft_consensus.cc:729] T 0d47a7a1285948468c0e6e1bddcd26d4 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bafdff0fd60146539631a476c4257e16, State: Initialized, Role: FOLLOWER
I20260807 19:42:09.605098 7624 ts_tablet_manager.cc:1428] T 0d47a7a1285948468c0e6e1bddcd26d4 P 051d464d644a4d4486462dfb18274c6a: Time spent starting tablet: real 0.015s user 0.005s sys 0.000s
I20260807 19:42:09.605708 7629 raft_consensus.cc:686] T d328d2e772ed4a6bb8597658f907f0f0 P 051d464d644a4d4486462dfb18274c6a [term 1 LEADER]: Becoming Leader. State: Replica: 051d464d644a4d4486462dfb18274c6a, State: Running, Role: LEADER
I20260807 19:42:09.605846 7620 consensus_queue.cc:260] T 0d47a7a1285948468c0e6e1bddcd26d4 P bafdff0fd60146539631a476c4257e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:09.606206 7629 consensus_queue.cc:237] T d328d2e772ed4a6bb8597658f907f0f0 P 051d464d644a4d4486462dfb18274c6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:09.609308 7620 ts_tablet_manager.cc:1428] T 0d47a7a1285948468c0e6e1bddcd26d4 P bafdff0fd60146539631a476c4257e16: Time spent starting tablet: real 0.025s user 0.002s sys 0.001s
I20260807 19:42:09.609724 7620 tablet_bootstrap.cc:492] T 9a45d0765a2c4d93ac9448496ce9c325 P bafdff0fd60146539631a476c4257e16: Bootstrap starting.
I20260807 19:42:09.611774 6991 catalog_manager.cc:4841] TS bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225): tablet 1512c8407da64adca2d3019f9b8ed1bd (table default.table_c4966a160ae2456ebe5e58c41a9ac129 [id=490e9d5bbd5347aaae98c3a81cb3fe1e]) successfully deleted
I20260807 19:42:09.613186 7620 tablet_bootstrap.cc:654] T 9a45d0765a2c4d93ac9448496ce9c325 P bafdff0fd60146539631a476c4257e16: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:09.617748 7624 tablet_bootstrap.cc:492] T 9a45d0765a2c4d93ac9448496ce9c325 P 051d464d644a4d4486462dfb18274c6a: Bootstrap starting.
I20260807 19:42:09.619059 7620 tablet_bootstrap.cc:492] T 9a45d0765a2c4d93ac9448496ce9c325 P bafdff0fd60146539631a476c4257e16: No bootstrap required, opened a new log
I20260807 19:42:09.619246 7620 ts_tablet_manager.cc:1397] T 9a45d0765a2c4d93ac9448496ce9c325 P bafdff0fd60146539631a476c4257e16: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.000s
I20260807 19:42:09.619669 7624 tablet_bootstrap.cc:654] T 9a45d0765a2c4d93ac9448496ce9c325 P 051d464d644a4d4486462dfb18274c6a: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:09.620142 7620 raft_consensus.cc:348] T 9a45d0765a2c4d93ac9448496ce9c325 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:09.620404 7620 raft_consensus.cc:374] T 9a45d0765a2c4d93ac9448496ce9c325 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:09.620509 7620 raft_consensus.cc:729] T 9a45d0765a2c4d93ac9448496ce9c325 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bafdff0fd60146539631a476c4257e16, State: Initialized, Role: FOLLOWER
I20260807 19:42:09.620743 7620 consensus_queue.cc:260] T 9a45d0765a2c4d93ac9448496ce9c325 P bafdff0fd60146539631a476c4257e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:09.623917 7620 ts_tablet_manager.cc:1428] T 9a45d0765a2c4d93ac9448496ce9c325 P bafdff0fd60146539631a476c4257e16: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260807 19:42:09.627830 7622 tablet_bootstrap.cc:492] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da: No bootstrap required, opened a new log
I20260807 19:42:09.628050 7622 ts_tablet_manager.cc:1397] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da: Time spent bootstrapping tablet: real 0.060s user 0.003s sys 0.003s
I20260807 19:42:09.628928 7622 raft_consensus.cc:348] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:09.629211 7622 raft_consensus.cc:374] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:09.629294 7622 raft_consensus.cc:729] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35b5ecc2c9ef47c2bbe35653633e14da, State: Initialized, Role: FOLLOWER
I20260807 19:42:09.629565 7622 consensus_queue.cc:260] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:09.632055 7622 ts_tablet_manager.cc:1428] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260807 19:42:09.633857 7622 tablet_bootstrap.cc:492] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da: Bootstrap starting.
I20260807 19:42:09.634064 7641 ts_tablet_manager.cc:1918] T 1512c8407da64adca2d3019f9b8ed1bd P 051d464d644a4d4486462dfb18274c6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260807 19:42:09.634268 7641 log.cc:1196] T 1512c8407da64adca2d3019f9b8ed1bd P 051d464d644a4d4486462dfb18274c6a: Deleting WAL directory at /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-0/wal/wals/1512c8407da64adca2d3019f9b8ed1bd
I20260807 19:42:09.634397 7643 ts_tablet_manager.cc:1939] T 1512c8407da64adca2d3019f9b8ed1bd P 35b5ecc2c9ef47c2bbe35653633e14da: Deleting consensus metadata
I20260807 19:42:09.634790 7641 ts_tablet_manager.cc:1939] T 1512c8407da64adca2d3019f9b8ed1bd P 051d464d644a4d4486462dfb18274c6a: Deleting consensus metadata
I20260807 19:42:09.636348 7643 tablet_replica.cc:330] T c2f7996c00ea44e8a1b8012ab53e52e2 P 35b5ecc2c9ef47c2bbe35653633e14da: stopping tablet replica
I20260807 19:42:09.636351 7641 tablet_replica.cc:330] T c2f7996c00ea44e8a1b8012ab53e52e2 P 051d464d644a4d4486462dfb18274c6a: stopping tablet replica
I20260807 19:42:09.636435 6991 catalog_manager.cc:4841] TS 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): tablet 1512c8407da64adca2d3019f9b8ed1bd (table default.table_c4966a160ae2456ebe5e58c41a9ac129 [id=490e9d5bbd5347aaae98c3a81cb3fe1e]) successfully deleted
I20260807 19:42:09.636559 7641 raft_consensus.cc:2229] T c2f7996c00ea44e8a1b8012ab53e52e2 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:09.636559 7643 raft_consensus.cc:2229] T c2f7996c00ea44e8a1b8012ab53e52e2 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:09.636740 7641 raft_consensus.cc:2258] T c2f7996c00ea44e8a1b8012ab53e52e2 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:09.636740 7643 raft_consensus.cc:2258] T c2f7996c00ea44e8a1b8012ab53e52e2 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:09.636759 6991 catalog_manager.cc:4841] TS 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483): tablet 1512c8407da64adca2d3019f9b8ed1bd (table default.table_c4966a160ae2456ebe5e58c41a9ac129 [id=490e9d5bbd5347aaae98c3a81cb3fe1e]) successfully deleted
I20260807 19:42:09.637818 7622 tablet_bootstrap.cc:654] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:09.639474 7642 tablet_replica.cc:330] T c2f7996c00ea44e8a1b8012ab53e52e2 P bafdff0fd60146539631a476c4257e16: stopping tablet replica
I20260807 19:42:09.639731 7642 raft_consensus.cc:2229] T c2f7996c00ea44e8a1b8012ab53e52e2 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:09.639940 7642 raft_consensus.cc:2258] T c2f7996c00ea44e8a1b8012ab53e52e2 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:09.641404 7624 tablet_bootstrap.cc:492] T 9a45d0765a2c4d93ac9448496ce9c325 P 051d464d644a4d4486462dfb18274c6a: No bootstrap required, opened a new log
I20260807 19:42:09.641566 7624 ts_tablet_manager.cc:1397] T 9a45d0765a2c4d93ac9448496ce9c325 P 051d464d644a4d4486462dfb18274c6a: Time spent bootstrapping tablet: real 0.024s user 0.000s sys 0.003s
I20260807 19:42:09.641700 7641 ts_tablet_manager.cc:1905] T c2f7996c00ea44e8a1b8012ab53e52e2 P 051d464d644a4d4486462dfb18274c6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:09.642437 7642 ts_tablet_manager.cc:1905] T c2f7996c00ea44e8a1b8012ab53e52e2 P bafdff0fd60146539631a476c4257e16: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:09.642449 7624 raft_consensus.cc:348] T 9a45d0765a2c4d93ac9448496ce9c325 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:09.642733 7624 raft_consensus.cc:374] T 9a45d0765a2c4d93ac9448496ce9c325 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:09.642848 7624 raft_consensus.cc:729] T 9a45d0765a2c4d93ac9448496ce9c325 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 051d464d644a4d4486462dfb18274c6a, State: Initialized, Role: FOLLOWER
I20260807 19:42:09.643283 7624 consensus_queue.cc:260] T 9a45d0765a2c4d93ac9448496ce9c325 P 051d464d644a4d4486462dfb18274c6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:09.643877 7624 ts_tablet_manager.cc:1428] T 9a45d0765a2c4d93ac9448496ce9c325 P 051d464d644a4d4486462dfb18274c6a: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260807 19:42:09.644302 7622 tablet_bootstrap.cc:492] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da: No bootstrap required, opened a new log
I20260807 19:42:09.645087 7622 ts_tablet_manager.cc:1397] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.002s
I20260807 19:42:09.646052 7622 raft_consensus.cc:348] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
W20260807 19:42:09.646937 7264 tablet.cc:2369] T 0d47a7a1285948468c0e6e1bddcd26d4 P 051d464d644a4d4486462dfb18274c6a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:42:09.650374 7622 raft_consensus.cc:374] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:09.650658 7622 raft_consensus.cc:729] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35b5ecc2c9ef47c2bbe35653633e14da, State: Initialized, Role: FOLLOWER
I20260807 19:42:09.651019 7622 consensus_queue.cc:260] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:09.651762 7622 ts_tablet_manager.cc:1428] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260807 19:42:09.654630 7642 ts_tablet_manager.cc:1918] T c2f7996c00ea44e8a1b8012ab53e52e2 P bafdff0fd60146539631a476c4257e16: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260807 19:42:09.654958 7642 log.cc:1196] T c2f7996c00ea44e8a1b8012ab53e52e2 P bafdff0fd60146539631a476c4257e16: Deleting WAL directory at /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-2/wal/wals/c2f7996c00ea44e8a1b8012ab53e52e2
I20260807 19:42:09.655450 7642 ts_tablet_manager.cc:1939] T c2f7996c00ea44e8a1b8012ab53e52e2 P bafdff0fd60146539631a476c4257e16: Deleting consensus metadata
I20260807 19:42:09.655922 7643 ts_tablet_manager.cc:1905] T c2f7996c00ea44e8a1b8012ab53e52e2 P 35b5ecc2c9ef47c2bbe35653633e14da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:09.656770 6991 catalog_manager.cc:4841] TS bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225): tablet c2f7996c00ea44e8a1b8012ab53e52e2 (table default.table_49a9f7854b4c4847bce27e16296322b5 [id=cf4da31cb40447199dc079b8b6e709bc]) successfully deleted
I20260807 19:42:09.659178 7641 ts_tablet_manager.cc:1918] T c2f7996c00ea44e8a1b8012ab53e52e2 P 051d464d644a4d4486462dfb18274c6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260807 19:42:09.659366 7641 log.cc:1196] T c2f7996c00ea44e8a1b8012ab53e52e2 P 051d464d644a4d4486462dfb18274c6a: Deleting WAL directory at /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-0/wal/wals/c2f7996c00ea44e8a1b8012ab53e52e2
I20260807 19:42:09.659775 7641 ts_tablet_manager.cc:1939] T c2f7996c00ea44e8a1b8012ab53e52e2 P 051d464d644a4d4486462dfb18274c6a: Deleting consensus metadata
I20260807 19:42:09.660954 6991 catalog_manager.cc:4841] TS 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): tablet c2f7996c00ea44e8a1b8012ab53e52e2 (table default.table_49a9f7854b4c4847bce27e16296322b5 [id=cf4da31cb40447199dc079b8b6e709bc]) successfully deleted
I20260807 19:42:09.666989 7352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "248c1670db7244cfac99dd4817b513a4" candidate_uuid: "bafdff0fd60146539631a476c4257e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" is_pre_election: true
I20260807 19:42:09.667316 7352 raft_consensus.cc:2454] T 248c1670db7244cfac99dd4817b513a4 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bafdff0fd60146539631a476c4257e16 in term 0.
I20260807 19:42:09.667981 7436 leader_election.cc:302] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35b5ecc2c9ef47c2bbe35653633e14da, bafdff0fd60146539631a476c4257e16; no voters:
I20260807 19:42:09.668431 7627 raft_consensus.cc:2781] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:42:09.668572 7627 raft_consensus.cc:482] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:42:09.668671 7627 raft_consensus.cc:3037] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:09.670465 7208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "248c1670db7244cfac99dd4817b513a4" candidate_uuid: "bafdff0fd60146539631a476c4257e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051d464d644a4d4486462dfb18274c6a" is_pre_election: true
I20260807 19:42:09.670969 7208 raft_consensus.cc:2454] T 248c1670db7244cfac99dd4817b513a4 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bafdff0fd60146539631a476c4257e16 in term 0.
I20260807 19:42:09.671321 7643 ts_tablet_manager.cc:1918] T c2f7996c00ea44e8a1b8012ab53e52e2 P 35b5ecc2c9ef47c2bbe35653633e14da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260807 19:42:09.671490 7643 log.cc:1196] T c2f7996c00ea44e8a1b8012ab53e52e2 P 35b5ecc2c9ef47c2bbe35653633e14da: Deleting WAL directory at /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-1/wal/wals/c2f7996c00ea44e8a1b8012ab53e52e2
I20260807 19:42:09.671875 7643 ts_tablet_manager.cc:1939] T c2f7996c00ea44e8a1b8012ab53e52e2 P 35b5ecc2c9ef47c2bbe35653633e14da: Deleting consensus metadata
I20260807 19:42:09.671711 7627 raft_consensus.cc:504] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:09.672387 7627 leader_election.cc:288] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 [CANDIDATE]: Term 1 election: Requested vote from peers 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467), 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483)
I20260807 19:42:09.672899 7352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "248c1670db7244cfac99dd4817b513a4" candidate_uuid: "bafdff0fd60146539631a476c4257e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35b5ecc2c9ef47c2bbe35653633e14da"
I20260807 19:42:09.673026 6991 catalog_manager.cc:4841] TS 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483): tablet c2f7996c00ea44e8a1b8012ab53e52e2 (table default.table_49a9f7854b4c4847bce27e16296322b5 [id=cf4da31cb40447199dc079b8b6e709bc]) successfully deleted
I20260807 19:42:09.673151 7352 raft_consensus.cc:3037] T 248c1670db7244cfac99dd4817b513a4 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:09.675896 7352 raft_consensus.cc:2454] T 248c1670db7244cfac99dd4817b513a4 P 35b5ecc2c9ef47c2bbe35653633e14da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bafdff0fd60146539631a476c4257e16 in term 1.
I20260807 19:42:09.676458 7436 leader_election.cc:302] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35b5ecc2c9ef47c2bbe35653633e14da, bafdff0fd60146539631a476c4257e16; no voters:
I20260807 19:42:09.677217 7627 raft_consensus.cc:2781] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:09.677346 7208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "248c1670db7244cfac99dd4817b513a4" candidate_uuid: "bafdff0fd60146539631a476c4257e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051d464d644a4d4486462dfb18274c6a"
I20260807 19:42:09.677703 7208 raft_consensus.cc:3037] T 248c1670db7244cfac99dd4817b513a4 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:09.681530 7208 raft_consensus.cc:2454] T 248c1670db7244cfac99dd4817b513a4 P 051d464d644a4d4486462dfb18274c6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bafdff0fd60146539631a476c4257e16 in term 1.
I20260807 19:42:09.687784 7627 raft_consensus.cc:686] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 [term 1 LEADER]: Becoming Leader. State: Replica: bafdff0fd60146539631a476c4257e16, State: Running, Role: LEADER
I20260807 19:42:09.688303 7627 consensus_queue.cc:237] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
19:42:09.781 [HMS - ERROR - pool-6-thread-1] (RetryingHMSHandler.java:201) AlreadyExistsException(message:Table hive.default.table_93e5970d7fe54e40903e703a8eecd0cf already exists)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1839)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1786)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_with_environment_context(HiveMetaStore.java:2035)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
at com.sun.proxy.$Proxy25.create_table_with_environment_context(Unknown Source)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:14800)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:14784)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:104)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
W20260807 19:42:09.795377 6999 catalog_manager.cc:2144] Already present: failed to create HMS catalog entry for table [id=12c16662323948498e5ad2fada518ede]: failed to create Hive MetaStore table: TException - service has thrown: AlreadyExistsException(message=Table hive.default.table_93e5970d7fe54e40903e703a8eecd0cf already exists)
I20260807 19:42:09.798338 6999 catalog_manager.cc:5495] T d328d2e772ed4a6bb8597658f907f0f0 P 051d464d644a4d4486462dfb18274c6a reported cstate change: term changed from 0 to 1, leader changed from <none> to 051d464d644a4d4486462dfb18274c6a (127.6.109.1). New cstate: current_term: 1 leader_uuid: "051d464d644a4d4486462dfb18274c6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } health_report { overall_health: UNKNOWN } } }
I20260807 19:42:09.809183 6999 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37396:
name: "default.Table_5d300b1b9ec343b88bbb7481707c963a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:09.830679 7630 raft_consensus.cc:482] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:09.830972 7630 raft_consensus.cc:504] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:09.832983 7630 leader_election.cc:288] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
I20260807 19:42:09.844904 7208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a45d0765a2c4d93ac9448496ce9c325" candidate_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051d464d644a4d4486462dfb18274c6a" is_pre_election: true
I20260807 19:42:09.845275 7208 raft_consensus.cc:2454] T 9a45d0765a2c4d93ac9448496ce9c325 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35b5ecc2c9ef47c2bbe35653633e14da in term 0.
I20260807 19:42:09.846025 7287 leader_election.cc:302] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051d464d644a4d4486462dfb18274c6a, 35b5ecc2c9ef47c2bbe35653633e14da; no voters:
I20260807 19:42:09.846654 7630 raft_consensus.cc:2781] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:42:09.846819 7630 raft_consensus.cc:482] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:42:09.846906 7630 raft_consensus.cc:3037] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:09.850425 7501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a45d0765a2c4d93ac9448496ce9c325" candidate_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bafdff0fd60146539631a476c4257e16" is_pre_election: true
I20260807 19:42:09.850844 7501 raft_consensus.cc:2454] T 9a45d0765a2c4d93ac9448496ce9c325 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35b5ecc2c9ef47c2bbe35653633e14da in term 0.
I20260807 19:42:09.853835 7630 raft_consensus.cc:504] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:09.855840 7208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a45d0765a2c4d93ac9448496ce9c325" candidate_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051d464d644a4d4486462dfb18274c6a"
I20260807 19:42:09.856132 7208 raft_consensus.cc:3037] T 9a45d0765a2c4d93ac9448496ce9c325 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:09.859277 7208 raft_consensus.cc:2454] T 9a45d0765a2c4d93ac9448496ce9c325 P 051d464d644a4d4486462dfb18274c6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35b5ecc2c9ef47c2bbe35653633e14da in term 1.
I20260807 19:42:09.859942 7287 leader_election.cc:302] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051d464d644a4d4486462dfb18274c6a, 35b5ecc2c9ef47c2bbe35653633e14da; no voters:
I20260807 19:42:09.861490 7630 leader_election.cc:288] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 election: Requested vote from peers 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
I20260807 19:42:09.861754 7630 raft_consensus.cc:2781] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:09.861912 7630 raft_consensus.cc:686] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da [term 1 LEADER]: Becoming Leader. State: Replica: 35b5ecc2c9ef47c2bbe35653633e14da, State: Running, Role: LEADER
I20260807 19:42:09.862315 7630 consensus_queue.cc:237] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:09.877024 7501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a45d0765a2c4d93ac9448496ce9c325" candidate_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bafdff0fd60146539631a476c4257e16"
I20260807 19:42:09.877333 7501 raft_consensus.cc:3037] T 9a45d0765a2c4d93ac9448496ce9c325 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:09.880435 7501 raft_consensus.cc:2454] T 9a45d0765a2c4d93ac9448496ce9c325 P bafdff0fd60146539631a476c4257e16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35b5ecc2c9ef47c2bbe35653633e14da in term 1.
I20260807 19:42:09.883705 7001 catalog_manager.cc:2148] created HMS catalog entry for table [id=bff436a0a60542b9b0fecacf4a6ed1b6]
I20260807 19:42:09.890470 7001 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:37396 (request call id 9) took 1015 ms. Trace:
I20260807 19:42:09.890682 7001 rpcz_store.cc:276] 0807 19:42:08.875400 (+ 0us) service_pool.cc:170] Inserting onto call queue
0807 19:42:08.877045 (+ 1645us) service_pool.cc:227] Handling call
0807 19:42:08.890995 (+ 13950us) catalog_manager.cc:2075] Acquired catalog manager lock
0807 19:42:08.891991 (+ 996us) catalog_manager.cc:2121] Created new table and tablet info
0807 19:42:09.883627 (+991636us) catalog_manager.cc:2147] Created new table in HMS catalog
0807 19:42:09.890144 (+ 6517us) catalog_manager.cc:2181] Wrote table and tablets to system table
0807 19:42:09.890281 (+ 137us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0807 19:42:09.890417 (+ 136us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0807 19:42:09.884592 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 19:42:09.884761 (+ 169us) write_op.cc:432] Acquiring schema lock in shared mode
0807 19:42:09.884769 (+ 8us) write_op.cc:435] Acquired schema lock
0807 19:42:09.884773 (+ 4us) tablet.cc:660] Decoding operations
0807 19:42:09.884903 (+ 130us) write_op.cc:620] Acquiring the partition lock for write op
0807 19:42:09.884922 (+ 19us) write_op.cc:641] Partition lock acquired for write op
0807 19:42:09.884930 (+ 8us) tablet.cc:683] Acquiring locks for 2 operations
0807 19:42:09.885049 (+ 119us) tablet.cc:699] Row locks acquired
0807 19:42:09.885052 (+ 3us) write_op.cc:260] PREPARE: finished
0807 19:42:09.885106 (+ 54us) write_op.cc:270] Start()
0807 19:42:09.885154 (+ 48us) write_op.cc:276] Timestamp: P: 1786131729885072 usec, L: 0
0807 19:42:09.885161 (+ 7us) op_driver.cc:331] REPLICATION: starting
0807 19:42:09.885439 (+ 278us) log.cc:842] Serialized 432 byte log entry
0807 19:42:09.888644 (+ 3205us) op_driver.cc:444] REPLICATION: finished
0807 19:42:09.888796 (+ 152us) write_op.cc:301] APPLY: starting
0807 19:42:09.888814 (+ 18us) tablet.cc:1365] starting BulkCheckPresence
0807 19:42:09.888861 (+ 47us) tablet.cc:1368] finished BulkCheckPresence
0807 19:42:09.888888 (+ 27us) tablet.cc:1370] starting ApplyRowOperation cycle
0807 19:42:09.889184 (+ 296us) tablet.cc:1381] finished ApplyRowOperation cycle
0807 19:42:09.889225 (+ 41us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0807 19:42:09.889231 (+ 6us) write_op.cc:312] APPLY: finished
0807 19:42:09.889339 (+ 108us) log.cc:842] Serialized 28 byte log entry
0807 19:42:09.889425 (+ 86us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:42:09.889451 (+ 26us) write_op.cc:454] Released schema lock
0807 19:42:09.889458 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0807 19:42:09.889527 (+ 69us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:42:09.889531 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":902861,"Hive Metastore.run_cpu_time_us":357,"Hive Metastore.run_wall_time_us":88237,"mutex_wait_us":850,"spinlock_wait_cycles":405888,"child_traces":[["op",{"apply.queue_time_us":98,"apply.run_cpu_time_us":792,"apply.run_wall_time_us":791,"mutex_wait_us":27,"num_ops":2,"prepare.queue_time_us":72,"prepare.run_cpu_time_us":1030,"prepare.run_wall_time_us":1031,"raft.queue_time_us":203,"raft.run_cpu_time_us":310,"raft.run_wall_time_us":311,"replication_time_us":3454,"spinlock_wait_cycles":2944}]]}
I20260807 19:42:09.892802 7001 catalog_manager.cc:5495] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 reported cstate change: term changed from 0 to 1, leader changed from <none> to bafdff0fd60146539631a476c4257e16 (127.6.109.3). New cstate: current_term: 1 leader_uuid: "bafdff0fd60146539631a476c4257e16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } health_report { overall_health: HEALTHY } } }
I20260807 19:42:09.911707 7480 tablet_service.cc:1467] Processing CreateTablet for tablet 807180c74470485182a926ab39d79ab4 (DEFAULT_TABLE table=default.table_ca953b708a5a408898f72c483fc31ecf [id=bff436a0a60542b9b0fecacf4a6ed1b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:09.912413 7480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 807180c74470485182a926ab39d79ab4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:09.912680 7001 catalog_manager.cc:5495] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da reported cstate change: term changed from 0 to 1, leader changed from <none> to 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2). New cstate: current_term: 1 leader_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } health_report { overall_health: UNKNOWN } } }
I20260807 19:42:09.914068 7187 tablet_service.cc:1467] Processing CreateTablet for tablet 807180c74470485182a926ab39d79ab4 (DEFAULT_TABLE table=default.table_ca953b708a5a408898f72c483fc31ecf [id=bff436a0a60542b9b0fecacf4a6ed1b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:09.914628 7187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 807180c74470485182a926ab39d79ab4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:09.924881 7620 tablet_bootstrap.cc:492] T 807180c74470485182a926ab39d79ab4 P bafdff0fd60146539631a476c4257e16: Bootstrap starting.
I20260807 19:42:09.926910 7620 tablet_bootstrap.cc:654] T 807180c74470485182a926ab39d79ab4 P bafdff0fd60146539631a476c4257e16: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:09.930052 7624 tablet_bootstrap.cc:492] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a: Bootstrap starting.
I20260807 19:42:09.930658 7331 tablet_service.cc:1467] Processing CreateTablet for tablet 807180c74470485182a926ab39d79ab4 (DEFAULT_TABLE table=default.table_ca953b708a5a408898f72c483fc31ecf [id=bff436a0a60542b9b0fecacf4a6ed1b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:09.931272 7331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 807180c74470485182a926ab39d79ab4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:09.931905 7624 tablet_bootstrap.cc:654] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:09.943310 7001 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37396:
table { table_name: "default.Table_472c6a33c87c4f1b9cdb926a2e5a6344" } new_table_name: "default.Table_cda45f853cca4f52a3371bbaec9c7750" modify_external_catalogs: true
I20260807 19:42:09.949770 7622 tablet_bootstrap.cc:492] T 807180c74470485182a926ab39d79ab4 P 35b5ecc2c9ef47c2bbe35653633e14da: Bootstrap starting.
I20260807 19:42:09.951875 7622 tablet_bootstrap.cc:654] T 807180c74470485182a926ab39d79ab4 P 35b5ecc2c9ef47c2bbe35653633e14da: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:09.953418 6997 catalog_manager.cc:2148] created HMS catalog entry for table [id=9918b1d3805d424b92055cf267b8a4ab]
I20260807 19:42:09.956633 7622 tablet_bootstrap.cc:492] T 807180c74470485182a926ab39d79ab4 P 35b5ecc2c9ef47c2bbe35653633e14da: No bootstrap required, opened a new log
I20260807 19:42:09.956956 7622 ts_tablet_manager.cc:1397] T 807180c74470485182a926ab39d79ab4 P 35b5ecc2c9ef47c2bbe35653633e14da: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
W20260807 19:42:09.959203 7003 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_b6eca8c0167f4d80b6ef4f8125aea68f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260807 19:42:09.960481 7004 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_8cc3d0e85b19426f8df95f1894914926 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260807 19:42:09.961284 6995 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_61514c4f10b54148a57bbaf295205ace in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:42:09.963518 7622 raft_consensus.cc:348] T 807180c74470485182a926ab39d79ab4 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } }
I20260807 19:42:09.963997 7622 raft_consensus.cc:374] T 807180c74470485182a926ab39d79ab4 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:09.964128 7622 raft_consensus.cc:729] T 807180c74470485182a926ab39d79ab4 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35b5ecc2c9ef47c2bbe35653633e14da, State: Initialized, Role: FOLLOWER
I20260807 19:42:09.964504 7622 consensus_queue.cc:260] T 807180c74470485182a926ab39d79ab4 P 35b5ecc2c9ef47c2bbe35653633e14da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } }
I20260807 19:42:09.969281 6997 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:37396 (request call id 13) took 1093 ms. Trace:
I20260807 19:42:09.969444 6997 rpcz_store.cc:276] 0807 19:42:08.875910 (+ 0us) service_pool.cc:170] Inserting onto call queue
0807 19:42:08.878691 (+ 2781us) service_pool.cc:227] Handling call
0807 19:42:08.890567 (+ 11876us) catalog_manager.cc:2075] Acquired catalog manager lock
0807 19:42:08.892953 (+ 2386us) catalog_manager.cc:2121] Created new table and tablet info
0807 19:42:09.953360 (+1060407us) catalog_manager.cc:2147] Created new table in HMS catalog
0807 19:42:09.968951 (+ 15591us) catalog_manager.cc:2181] Wrote table and tablets to system table
0807 19:42:09.969098 (+ 147us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0807 19:42:09.969223 (+ 125us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0807 19:42:09.954406 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 19:42:09.954566 (+ 160us) write_op.cc:432] Acquiring schema lock in shared mode
0807 19:42:09.954572 (+ 6us) write_op.cc:435] Acquired schema lock
0807 19:42:09.954574 (+ 2us) tablet.cc:660] Decoding operations
0807 19:42:09.954681 (+ 107us) write_op.cc:620] Acquiring the partition lock for write op
0807 19:42:09.954695 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0807 19:42:09.954700 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
0807 19:42:09.954796 (+ 96us) tablet.cc:699] Row locks acquired
0807 19:42:09.954809 (+ 13us) write_op.cc:260] PREPARE: finished
0807 19:42:09.954859 (+ 50us) write_op.cc:270] Start()
0807 19:42:09.954901 (+ 42us) write_op.cc:276] Timestamp: P: 1786131729954819 usec, L: 0
0807 19:42:09.954905 (+ 4us) op_driver.cc:331] REPLICATION: starting
0807 19:42:09.955128 (+ 223us) log.cc:842] Serialized 432 byte log entry
0807 19:42:09.963467 (+ 8339us) op_driver.cc:444] REPLICATION: finished
0807 19:42:09.966027 (+ 2560us) write_op.cc:301] APPLY: starting
0807 19:42:09.966046 (+ 19us) tablet.cc:1365] starting BulkCheckPresence
0807 19:42:09.966093 (+ 47us) tablet.cc:1368] finished BulkCheckPresence
0807 19:42:09.966150 (+ 57us) tablet.cc:1370] starting ApplyRowOperation cycle
0807 19:42:09.966261 (+ 111us) tablet.cc:1381] finished ApplyRowOperation cycle
0807 19:42:09.966299 (+ 38us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0807 19:42:09.966305 (+ 6us) write_op.cc:312] APPLY: finished
0807 19:42:09.966528 (+ 223us) log.cc:842] Serialized 28 byte log entry
0807 19:42:09.966888 (+ 360us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:42:09.966915 (+ 27us) write_op.cc:454] Released schema lock
0807 19:42:09.966924 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0807 19:42:09.966989 (+ 65us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:42:09.966993 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":990516,"Hive Metastore.run_cpu_time_us":296,"Hive Metastore.run_wall_time_us":69548,"mutex_wait_us":265,"spinlock_wait_cycles":59264,"child_traces":[["op",{"apply.queue_time_us":2481,"apply.run_cpu_time_us":765,"apply.run_wall_time_us":1035,"num_ops":2,"prepare.queue_time_us":63,"prepare.run_cpu_time_us":893,"prepare.run_wall_time_us":893,"raft.queue_time_us":1837,"raft.run_cpu_time_us":278,"raft.run_wall_time_us":279,"replication_time_us":8532}]]}
I20260807 19:42:09.978720 7187 tablet_service.cc:1467] Processing CreateTablet for tablet 769aecb6d5ca42e09edba0277a022901 (DEFAULT_TABLE table=default.table_5d5e7f4976094d7a942e0ba75696c8f7 [id=9918b1d3805d424b92055cf267b8a4ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:09.979274 7187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 769aecb6d5ca42e09edba0277a022901. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:09.980753 6997 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37396:
name: "default.Table_f2d1ca595d0c43ec982c1377f39155ee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:09.982412 7622 ts_tablet_manager.cc:1428] T 807180c74470485182a926ab39d79ab4 P 35b5ecc2c9ef47c2bbe35653633e14da: Time spent starting tablet: real 0.025s user 0.002s sys 0.000s
I20260807 19:42:09.984349 7629 raft_consensus.cc:482] T 0d47a7a1285948468c0e6e1bddcd26d4 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:09.984544 7629 raft_consensus.cc:504] T 0d47a7a1285948468c0e6e1bddcd26d4 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:09.985248 7629 leader_election.cc:288] T 0d47a7a1285948468c0e6e1bddcd26d4 P 051d464d644a4d4486462dfb18274c6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
I20260807 19:42:09.986127 7501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d47a7a1285948468c0e6e1bddcd26d4" candidate_uuid: "051d464d644a4d4486462dfb18274c6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bafdff0fd60146539631a476c4257e16" is_pre_election: true
I20260807 19:42:09.986397 7501 raft_consensus.cc:2454] T 0d47a7a1285948468c0e6e1bddcd26d4 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 051d464d644a4d4486462dfb18274c6a in term 0.
F20260807 19:42:09.986824 7060 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_c4966a160ae2456ebe5e58c41a9ac129
*** Check failure stack trace: ***
I20260807 19:42:09.987975 7480 tablet_service.cc:1467] Processing CreateTablet for tablet 769aecb6d5ca42e09edba0277a022901 (DEFAULT_TABLE table=default.table_5d5e7f4976094d7a942e0ba75696c8f7 [id=9918b1d3805d424b92055cf267b8a4ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:09.988488 7480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 769aecb6d5ca42e09edba0277a022901. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:09.990180 7352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d47a7a1285948468c0e6e1bddcd26d4" candidate_uuid: "051d464d644a4d4486462dfb18274c6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" is_pre_election: true
I20260807 19:42:09.990432 7352 raft_consensus.cc:2454] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 051d464d644a4d4486462dfb18274c6a in term 0.
I20260807 19:42:09.991030 7140 leader_election.cc:302] T 0d47a7a1285948468c0e6e1bddcd26d4 P 051d464d644a4d4486462dfb18274c6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051d464d644a4d4486462dfb18274c6a, bafdff0fd60146539631a476c4257e16; no voters:
I20260807 19:42:09.991868 7629 raft_consensus.cc:2781] T 0d47a7a1285948468c0e6e1bddcd26d4 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:42:09.991979 7629 raft_consensus.cc:482] T 0d47a7a1285948468c0e6e1bddcd26d4 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:42:09.992086 7629 raft_consensus.cc:3037] T 0d47a7a1285948468c0e6e1bddcd26d4 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:09.994084 7331 tablet_service.cc:1467] Processing CreateTablet for tablet 769aecb6d5ca42e09edba0277a022901 (DEFAULT_TABLE table=default.table_5d5e7f4976094d7a942e0ba75696c8f7 [id=9918b1d3805d424b92055cf267b8a4ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:09.994603 7331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 769aecb6d5ca42e09edba0277a022901. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:09.994915 7629 raft_consensus.cc:504] T 0d47a7a1285948468c0e6e1bddcd26d4 P 051d464d644a4d4486462dfb18274c6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:09.996234 7352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d47a7a1285948468c0e6e1bddcd26d4" candidate_uuid: "051d464d644a4d4486462dfb18274c6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35b5ecc2c9ef47c2bbe35653633e14da"
I20260807 19:42:09.996486 7352 raft_consensus.cc:3037] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:09.999884 7352 raft_consensus.cc:2454] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 051d464d644a4d4486462dfb18274c6a in term 1.
I20260807 19:42:10.001775 7624 tablet_bootstrap.cc:492] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a: No bootstrap required, opened a new log
@ 0x7f967a40a9ad google::LogMessage::Fail() at ??:0
I20260807 19:42:10.002058 7624 ts_tablet_manager.cc:1397] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a: Time spent bootstrapping tablet: real 0.072s user 0.019s sys 0.019s
I20260807 19:42:10.003340 7624 raft_consensus.cc:348] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } }
I20260807 19:42:10.003669 7624 raft_consensus.cc:374] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:10.003775 7624 raft_consensus.cc:729] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 051d464d644a4d4486462dfb18274c6a, State: Initialized, Role: FOLLOWER
I20260807 19:42:10.003842 7140 leader_election.cc:302] T 0d47a7a1285948468c0e6e1bddcd26d4 P 051d464d644a4d4486462dfb18274c6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051d464d644a4d4486462dfb18274c6a, 35b5ecc2c9ef47c2bbe35653633e14da; no voters:
I20260807 19:42:10.004221 7646 raft_consensus.cc:2781] T 0d47a7a1285948468c0e6e1bddcd26d4 P 051d464d644a4d4486462dfb18274c6a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:10.004067 7624 consensus_queue.cc:260] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } }
I20260807 19:42:10.004405 7646 raft_consensus.cc:686] T 0d47a7a1285948468c0e6e1bddcd26d4 P 051d464d644a4d4486462dfb18274c6a [term 1 LEADER]: Becoming Leader. State: Replica: 051d464d644a4d4486462dfb18274c6a, State: Running, Role: LEADER
I20260807 19:42:10.004741 7646 consensus_queue.cc:237] T 0d47a7a1285948468c0e6e1bddcd26d4 P 051d464d644a4d4486462dfb18274c6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
@ 0x7f967a40d88a google::LogMessage::SendToLog() at ??:0
@ 0x7f967a40a4de google::LogMessage::Flush() at ??:0
I20260807 19:42:10.010730 7501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d47a7a1285948468c0e6e1bddcd26d4" candidate_uuid: "051d464d644a4d4486462dfb18274c6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bafdff0fd60146539631a476c4257e16"
I20260807 19:42:10.011023 7501 raft_consensus.cc:3037] T 0d47a7a1285948468c0e6e1bddcd26d4 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:10.014038 7501 raft_consensus.cc:2454] T 0d47a7a1285948468c0e6e1bddcd26d4 P bafdff0fd60146539631a476c4257e16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 051d464d644a4d4486462dfb18274c6a in term 1.
I20260807 19:42:10.015587 7620 tablet_bootstrap.cc:492] T 807180c74470485182a926ab39d79ab4 P bafdff0fd60146539631a476c4257e16: No bootstrap required, opened a new log
I20260807 19:42:10.015905 7620 ts_tablet_manager.cc:1397] T 807180c74470485182a926ab39d79ab4 P bafdff0fd60146539631a476c4257e16: Time spent bootstrapping tablet: real 0.091s user 0.012s sys 0.001s
I20260807 19:42:10.016835 7620 raft_consensus.cc:348] T 807180c74470485182a926ab39d79ab4 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } }
I20260807 19:42:10.017127 7620 raft_consensus.cc:374] T 807180c74470485182a926ab39d79ab4 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:10.017210 7620 raft_consensus.cc:729] T 807180c74470485182a926ab39d79ab4 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bafdff0fd60146539631a476c4257e16, State: Initialized, Role: FOLLOWER
I20260807 19:42:10.018056 7629 leader_election.cc:288] T 0d47a7a1285948468c0e6e1bddcd26d4 P 051d464d644a4d4486462dfb18274c6a [CANDIDATE]: Term 1 election: Requested vote from peers 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
I20260807 19:42:10.019552 7620 consensus_queue.cc:260] T 807180c74470485182a926ab39d79ab4 P bafdff0fd60146539631a476c4257e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } }
I20260807 19:42:10.020262 7620 ts_tablet_manager.cc:1428] T 807180c74470485182a926ab39d79ab4 P bafdff0fd60146539631a476c4257e16: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260807 19:42:10.020560 7620 tablet_bootstrap.cc:492] T 769aecb6d5ca42e09edba0277a022901 P bafdff0fd60146539631a476c4257e16: Bootstrap starting.
@ 0x7f967a40a6d1 google::LogMessage::~LogMessage() at ??:0
I20260807 19:42:10.026878 7622 tablet_bootstrap.cc:492] T 769aecb6d5ca42e09edba0277a022901 P 35b5ecc2c9ef47c2bbe35653633e14da: Bootstrap starting.
I20260807 19:42:10.029978 7622 tablet_bootstrap.cc:654] T 769aecb6d5ca42e09edba0277a022901 P 35b5ecc2c9ef47c2bbe35653633e14da: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:10.033553 7624 ts_tablet_manager.cc:1428] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a: Time spent starting tablet: real 0.031s user 0.004s sys 0.005s
I20260807 19:42:10.035697 7624 tablet_bootstrap.cc:492] T 769aecb6d5ca42e09edba0277a022901 P 051d464d644a4d4486462dfb18274c6a: Bootstrap starting.
I20260807 19:42:10.037890 7624 tablet_bootstrap.cc:654] T 769aecb6d5ca42e09edba0277a022901 P 051d464d644a4d4486462dfb18274c6a: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:10.038956 7620 tablet_bootstrap.cc:654] T 769aecb6d5ca42e09edba0277a022901 P bafdff0fd60146539631a476c4257e16: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:10.042786 7622 tablet_bootstrap.cc:492] T 769aecb6d5ca42e09edba0277a022901 P 35b5ecc2c9ef47c2bbe35653633e14da: No bootstrap required, opened a new log
I20260807 19:42:10.042984 7622 ts_tablet_manager.cc:1397] T 769aecb6d5ca42e09edba0277a022901 P 35b5ecc2c9ef47c2bbe35653633e14da: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.000s
I20260807 19:42:10.044165 7622 raft_consensus.cc:348] T 769aecb6d5ca42e09edba0277a022901 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:10.044574 7622 raft_consensus.cc:374] T 769aecb6d5ca42e09edba0277a022901 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:10.044684 7622 raft_consensus.cc:729] T 769aecb6d5ca42e09edba0277a022901 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35b5ecc2c9ef47c2bbe35653633e14da, State: Initialized, Role: FOLLOWER
I20260807 19:42:10.045073 7622 consensus_queue.cc:260] T 769aecb6d5ca42e09edba0277a022901 P 35b5ecc2c9ef47c2bbe35653633e14da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:10.047384 7622 ts_tablet_manager.cc:1428] T 769aecb6d5ca42e09edba0277a022901 P 35b5ecc2c9ef47c2bbe35653633e14da: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
@ 0x7f968ffa8ba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260807 19:42:10.054726 7620 tablet_bootstrap.cc:492] T 769aecb6d5ca42e09edba0277a022901 P bafdff0fd60146539631a476c4257e16: No bootstrap required, opened a new log
I20260807 19:42:10.054945 7620 ts_tablet_manager.cc:1397] T 769aecb6d5ca42e09edba0277a022901 P bafdff0fd60146539631a476c4257e16: Time spent bootstrapping tablet: real 0.034s user 0.008s sys 0.002s
I20260807 19:42:10.056069 7620 raft_consensus.cc:348] T 769aecb6d5ca42e09edba0277a022901 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:10.056344 7620 raft_consensus.cc:374] T 769aecb6d5ca42e09edba0277a022901 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:10.056443 7620 raft_consensus.cc:729] T 769aecb6d5ca42e09edba0277a022901 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bafdff0fd60146539631a476c4257e16, State: Initialized, Role: FOLLOWER
I20260807 19:42:10.056828 7620 consensus_queue.cc:260] T 769aecb6d5ca42e09edba0277a022901 P bafdff0fd60146539631a476c4257e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
@ 0x7f968ffa7901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260807 19:42:10.061277 7620 ts_tablet_manager.cc:1428] T 769aecb6d5ca42e09edba0277a022901 P bafdff0fd60146539631a476c4257e16: Time spent starting tablet: real 0.006s user 0.002s sys 0.001s
I20260807 19:42:10.066200 7624 tablet_bootstrap.cc:492] T 769aecb6d5ca42e09edba0277a022901 P 051d464d644a4d4486462dfb18274c6a: No bootstrap required, opened a new log
I20260807 19:42:10.066381 7624 ts_tablet_manager.cc:1397] T 769aecb6d5ca42e09edba0277a022901 P 051d464d644a4d4486462dfb18274c6a: Time spent bootstrapping tablet: real 0.033s user 0.001s sys 0.010s
I20260807 19:42:10.067313 7624 raft_consensus.cc:348] T 769aecb6d5ca42e09edba0277a022901 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:10.067589 7624 raft_consensus.cc:374] T 769aecb6d5ca42e09edba0277a022901 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:10.067684 7624 raft_consensus.cc:729] T 769aecb6d5ca42e09edba0277a022901 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 051d464d644a4d4486462dfb18274c6a, State: Initialized, Role: FOLLOWER
@ 0x7f968ffae66c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260807 19:42:10.067991 7624 consensus_queue.cc:260] T 769aecb6d5ca42e09edba0277a022901 P 051d464d644a4d4486462dfb18274c6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:10.069847 7624 ts_tablet_manager.cc:1428] T 769aecb6d5ca42e09edba0277a022901 P 051d464d644a4d4486462dfb18274c6a: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
@ 0x7f968ffae4e9 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f967d034eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f968030e6db start_thread at ??:0
@ 0x7f96792a461f clone at ??:0
W20260807 19:42:10.144176 7640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:10.144519 7640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:10.144668 7640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:10.156199 7640 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:10.156350 7640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:10.156407 7640 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:10.156458 7640 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:10.169445 7640 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.109.20:40195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:38987
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.109.61:39263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.109.61
--webserver_port=37257
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:10.171020 7640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:10.173249 7640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:10.196727 7680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:10.197738 7681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:10.199433 7683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:10.200464 7640 server_base.cc:1034] running on GCE node
I20260807 19:42:10.201849 7640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:10.204315 7640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:10.205772 7640 hybrid_clock.cc:648] HybridClock initialized: now 1786131730205640 us; error 120 us; skew 500 ppm
I20260807 19:42:10.206321 7640 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:42:10.217310 7640 webserver.cc:460] Webserver started at http://127.6.109.61:37257/ using document root <none> and password file <none>
I20260807 19:42:10.218638 7640 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:10.218983 7640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:10.227955 7640 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.001s sys 0.004s
I20260807 19:42:10.232829 7693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:10.235046 7640 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260807 19:42:10.235307 7640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-1/data,/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-1/wal
uuid: "1b782e30a8d44e7b94f080aa6327134f"
format_stamp: "Formatted at 2026-08-07 19:42:04 on dist-test-slave-p3t5"
I20260807 19:42:10.236975 7640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:10.259267 7640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:10.260563 7640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:10.260938 7640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:10.288985 7640 rpc_server.cc:307] RPC server started. Bound to: 127.6.109.61:39263
I20260807 19:42:10.289023 7745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.109.61:39263 every 8 connection(s)
I20260807 19:42:10.292150 7640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-1/data/info.pb
I20260807 19:42:10.301256 6580 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskeMwJpl/build/asan/bin/kudu as pid 7640
I20260807 19:42:10.305859 7746 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:42:10.314975 7746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f: Bootstrap starting.
I20260807 19:42:10.320258 7663 raft_consensus.cc:482] T 769aecb6d5ca42e09edba0277a022901 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:10.320564 7663 raft_consensus.cc:504] T 769aecb6d5ca42e09edba0277a022901 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:10.322983 7352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "769aecb6d5ca42e09edba0277a022901" candidate_uuid: "051d464d644a4d4486462dfb18274c6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" is_pre_election: true
I20260807 19:42:10.323329 7352 raft_consensus.cc:2454] T 769aecb6d5ca42e09edba0277a022901 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 051d464d644a4d4486462dfb18274c6a in term 0.
I20260807 19:42:10.324182 7140 leader_election.cc:302] T 769aecb6d5ca42e09edba0277a022901 P 051d464d644a4d4486462dfb18274c6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051d464d644a4d4486462dfb18274c6a, 35b5ecc2c9ef47c2bbe35653633e14da; no voters:
I20260807 19:42:10.324671 7646 raft_consensus.cc:2781] T 769aecb6d5ca42e09edba0277a022901 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:42:10.324878 7646 raft_consensus.cc:482] T 769aecb6d5ca42e09edba0277a022901 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:42:10.325032 7646 raft_consensus.cc:3037] T 769aecb6d5ca42e09edba0277a022901 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:10.326280 7501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "769aecb6d5ca42e09edba0277a022901" candidate_uuid: "051d464d644a4d4486462dfb18274c6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bafdff0fd60146539631a476c4257e16" is_pre_election: true
I20260807 19:42:10.326712 7501 raft_consensus.cc:2454] T 769aecb6d5ca42e09edba0277a022901 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 051d464d644a4d4486462dfb18274c6a in term 0.
I20260807 19:42:10.330096 7746 log.cc:824] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:10.330608 7663 leader_election.cc:288] T 769aecb6d5ca42e09edba0277a022901 P 051d464d644a4d4486462dfb18274c6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
I20260807 19:42:10.331471 7646 raft_consensus.cc:504] T 769aecb6d5ca42e09edba0277a022901 P 051d464d644a4d4486462dfb18274c6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:10.333308 7352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "769aecb6d5ca42e09edba0277a022901" candidate_uuid: "051d464d644a4d4486462dfb18274c6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35b5ecc2c9ef47c2bbe35653633e14da"
I20260807 19:42:10.333714 7352 raft_consensus.cc:3037] T 769aecb6d5ca42e09edba0277a022901 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:10.338905 7501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "769aecb6d5ca42e09edba0277a022901" candidate_uuid: "051d464d644a4d4486462dfb18274c6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bafdff0fd60146539631a476c4257e16"
I20260807 19:42:10.339298 7501 raft_consensus.cc:3037] T 769aecb6d5ca42e09edba0277a022901 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:10.341717 7352 raft_consensus.cc:2454] T 769aecb6d5ca42e09edba0277a022901 P 35b5ecc2c9ef47c2bbe35653633e14da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 051d464d644a4d4486462dfb18274c6a in term 1.
I20260807 19:42:10.341948 7646 leader_election.cc:288] T 769aecb6d5ca42e09edba0277a022901 P 051d464d644a4d4486462dfb18274c6a [CANDIDATE]: Term 1 election: Requested vote from peers 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
I20260807 19:42:10.342850 7140 leader_election.cc:302] T 769aecb6d5ca42e09edba0277a022901 P 051d464d644a4d4486462dfb18274c6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051d464d644a4d4486462dfb18274c6a, 35b5ecc2c9ef47c2bbe35653633e14da; no voters:
I20260807 19:42:10.343555 7646 raft_consensus.cc:2781] T 769aecb6d5ca42e09edba0277a022901 P 051d464d644a4d4486462dfb18274c6a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:10.343792 7646 raft_consensus.cc:686] T 769aecb6d5ca42e09edba0277a022901 P 051d464d644a4d4486462dfb18274c6a [term 1 LEADER]: Becoming Leader. State: Replica: 051d464d644a4d4486462dfb18274c6a, State: Running, Role: LEADER
I20260807 19:42:10.344203 7646 consensus_queue.cc:237] T 769aecb6d5ca42e09edba0277a022901 P 051d464d644a4d4486462dfb18274c6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:10.343537 7501 raft_consensus.cc:2454] T 769aecb6d5ca42e09edba0277a022901 P bafdff0fd60146539631a476c4257e16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 051d464d644a4d4486462dfb18274c6a in term 1.
I20260807 19:42:10.352166 7746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:42:10.353534 7746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f: Bootstrap complete.
I20260807 19:42:10.359110 7262 heartbeater.cc:347] Connected to a master server at 127.6.109.61:39263
I20260807 19:42:10.369738 7746 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 4 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:10.371106 7746 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b782e30a8d44e7b94f080aa6327134f, State: Initialized, Role: FOLLOWER
I20260807 19:42:10.372015 7746 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:10.373386 7751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 4 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:10.374071 7751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:10.374837 7746 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:10.425387 7767 catalog_manager.cc:1259] Loaded cluster ID: a80f7b000fe149f3a73e545a28c78fda
I20260807 19:42:10.425674 7767 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f: loading cluster ID for follower catalog manager: success
W20260807 19:42:10.428601 7559 connection.cc:535] client connection to 127.6.109.60:43129 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 19:42:10.429414 7559 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.6.109.60:43129)
I20260807 19:42:10.429421 7646 raft_consensus.cc:482] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:10.429653 7646 raft_consensus.cc:504] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } }
W20260807 19:42:10.429922 7559 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.6.109.60:43129)
I20260807 19:42:10.430392 7646 leader_election.cc:288] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
W20260807 19:42:10.430452 7559 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.6.109.60:43129)
I20260807 19:42:10.430511 7767 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f: acquiring CA information for follower catalog manager: success
W20260807 19:42:10.430919 7559 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.6.109.60:43129)
I20260807 19:42:10.431550 7352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "807180c74470485182a926ab39d79ab4" candidate_uuid: "051d464d644a4d4486462dfb18274c6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" is_pre_election: true
I20260807 19:42:10.431840 7352 raft_consensus.cc:2454] T 807180c74470485182a926ab39d79ab4 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 051d464d644a4d4486462dfb18274c6a in term 0.
I20260807 19:42:10.432258 7501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "807180c74470485182a926ab39d79ab4" candidate_uuid: "051d464d644a4d4486462dfb18274c6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bafdff0fd60146539631a476c4257e16" is_pre_election: true
I20260807 19:42:10.432507 7501 raft_consensus.cc:2454] T 807180c74470485182a926ab39d79ab4 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 051d464d644a4d4486462dfb18274c6a in term 0.
I20260807 19:42:10.433030 7140 leader_election.cc:302] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051d464d644a4d4486462dfb18274c6a, bafdff0fd60146539631a476c4257e16; no voters:
I20260807 19:42:10.433411 7767 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 19:42:10.434096 7646 raft_consensus.cc:2781] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:42:10.434206 7646 raft_consensus.cc:482] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:42:10.434303 7646 raft_consensus.cc:3037] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:10.437312 7646 raft_consensus.cc:504] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } }
W20260807 19:42:10.441133 7775 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:42:10.436148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.109.60:43129 (local address 127.0.0.1:37410)
0807 19:42:10.436855 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:10.436880 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:10.437011 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:10.440680 (+ 3669us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":753,"threads_started":1}
W20260807 19:42:10.441223 7636 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:42:10.439884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.109.60:43129 (local address 127.6.109.1:60667)
0807 19:42:10.440142 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:10.440255 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":133}
I20260807 19:42:10.442974 7352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "807180c74470485182a926ab39d79ab4" candidate_uuid: "051d464d644a4d4486462dfb18274c6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35b5ecc2c9ef47c2bbe35653633e14da"
I20260807 19:42:10.443229 7352 raft_consensus.cc:3037] T 807180c74470485182a926ab39d79ab4 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Advancing to term 1
W20260807 19:42:10.443295 7649 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:42:10.438731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.109.60:43129 (local address 127.6.109.3:59699)
0807 19:42:10.439007 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:10.439035 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:10.439162 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:10.442402 (+ 3240us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":164}
W20260807 19:42:10.444564 7656 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:42:10.439058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.109.60:43129 (local address 127.6.109.2:56977)
0807 19:42:10.439407 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:10.439426 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:10.439538 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:10.443679 (+ 4141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":204}
I20260807 19:42:10.446768 7352 raft_consensus.cc:2454] T 807180c74470485182a926ab39d79ab4 P 35b5ecc2c9ef47c2bbe35653633e14da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 051d464d644a4d4486462dfb18274c6a in term 1.
I20260807 19:42:10.446864 7646 leader_election.cc:288] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a [CANDIDATE]: Term 1 election: Requested vote from peers 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
I20260807 19:42:10.447652 7501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "807180c74470485182a926ab39d79ab4" candidate_uuid: "051d464d644a4d4486462dfb18274c6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bafdff0fd60146539631a476c4257e16"
I20260807 19:42:10.449782 7501 raft_consensus.cc:3037] T 807180c74470485182a926ab39d79ab4 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:10.453102 7501 raft_consensus.cc:2454] T 807180c74470485182a926ab39d79ab4 P bafdff0fd60146539631a476c4257e16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 051d464d644a4d4486462dfb18274c6a in term 1.
I20260807 19:42:10.453727 7140 leader_election.cc:302] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051d464d644a4d4486462dfb18274c6a, bafdff0fd60146539631a476c4257e16; no voters:
I20260807 19:42:10.454698 7646 raft_consensus.cc:2781] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:10.456197 7646 raft_consensus.cc:686] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a [term 1 LEADER]: Becoming Leader. State: Replica: 051d464d644a4d4486462dfb18274c6a, State: Running, Role: LEADER
I20260807 19:42:10.456497 7646 consensus_queue.cc:237] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } }
I20260807 19:42:10.458873 7640 master_runner.cc:186] Error getting master registration for 127.6.109.60:43129: Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: connect: Connection refused (error 111),
I20260807 19:42:10.481314 7640 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:42:10.487017 7709 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "051d464d644a4d4486462dfb18274c6a" instance_seqno: 1786131726838357) as {username='slave'} at 127.6.109.1:48411; Asking this server to re-register.
I20260807 19:42:10.487902 7262 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:10.490051 7709 ts_manager.cc:194] Registered new tserver with Master: 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467)
I20260807 19:42:10.494048 7777 raft_consensus.cc:482] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7b31edda0c4a48489ad60fc59c44ed31)
I20260807 19:42:10.494324 7777 raft_consensus.cc:504] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:10.495177 7777 leader_election.cc:288] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263), 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129)
W20260807 19:42:10.498036 6846 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: connect: Connection refused (error 111) [suppressed 8 similar messages]
W20260807 19:42:10.502668 6846 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129): Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: connect: Connection refused (error 111)
I20260807 19:42:10.505576 7721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "959ac7807a8b42a59c61a23d4605c054" candidate_term: 6 candidate_status { last_received { term: 5 index: 25 } } ignore_live_leader: false dest_uuid: "1b782e30a8d44e7b94f080aa6327134f" is_pre_election: true
I20260807 19:42:10.506569 7721 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 959ac7807a8b42a59c61a23d4605c054 in term 4.
I20260807 19:42:10.507331 6846 leader_election.cc:302] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054; no voters: 7b31edda0c4a48489ad60fc59c44ed31
I20260807 19:42:10.507787 7777 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:42:10.507943 7777 raft_consensus.cc:482] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7b31edda0c4a48489ad60fc59c44ed31)
I20260807 19:42:10.508054 7777 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:42:10.509330 6828 master_runner.cc:186] Error getting master registration for 127.6.109.60:43129: Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: connect: Connection refused (error 111),
I20260807 19:42:10.509539 6828 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:42:10.510733 7777 raft_consensus.cc:504] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:10.511333 7777 leader_election.cc:288] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 6 election: Requested vote from peers 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263), 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129)
I20260807 19:42:10.511886 7720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "959ac7807a8b42a59c61a23d4605c054" candidate_term: 6 candidate_status { last_received { term: 5 index: 25 } } ignore_live_leader: false dest_uuid: "1b782e30a8d44e7b94f080aa6327134f"
I20260807 19:42:10.512213 7720 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 4 FOLLOWER]: Advancing to term 6
W20260807 19:42:10.517299 6846 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129): Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: connect: Connection refused (error 111)
I20260807 19:42:10.518649 7720 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 959ac7807a8b42a59c61a23d4605c054 in term 6.
I20260807 19:42:10.520409 6846 leader_election.cc:302] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054; no voters: 7b31edda0c4a48489ad60fc59c44ed31
I20260807 19:42:10.520776 7777 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 19:42:10.521263 7777 raft_consensus.cc:686] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 6 LEADER]: Becoming Leader. State: Replica: 959ac7807a8b42a59c61a23d4605c054, State: Running, Role: LEADER
I20260807 19:42:10.521855 7777 consensus_queue.cc:237] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 5.25, Last appended by leader: 25, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:10.523617 7781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 959ac7807a8b42a59c61a23d4605c054. Latest consensus state: current_term: 6 leader_uuid: "959ac7807a8b42a59c61a23d4605c054" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:10.524109 7781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:10.524914 7782 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:10.527833 7782 catalog_manager.cc:669] Loaded metadata for table default.table_472c6a33c87c4f1b9cdb926a2e5a6344 [id=0b2f82ef918b47d59b34a5b5757b9486]
I20260807 19:42:10.528450 7782 catalog_manager.cc:669] Loaded metadata for table default.table_c4966a160ae2456ebe5e58c41a9ac129 [id=490e9d5bbd5347aaae98c3a81cb3fe1e]
I20260807 19:42:10.528385 7627 consensus_queue.cc:1035] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:10.528971 7782 catalog_manager.cc:669] Loaded metadata for table default.table_5d5e7f4976094d7a942e0ba75696c8f7 [id=9918b1d3805d424b92055cf267b8a4ab]
I20260807 19:42:10.529392 7782 catalog_manager.cc:669] Loaded metadata for table default.table_271147f124bf45a88d332b72ac8d3e13 [id=a75161def1a34954bf0846ab30a8cc38]
I20260807 19:42:10.529839 7782 catalog_manager.cc:669] Loaded metadata for table default.table_ca953b708a5a408898f72c483fc31ecf [id=bff436a0a60542b9b0fecacf4a6ed1b6]
I20260807 19:42:10.530860 7782 catalog_manager.cc:669] Loaded metadata for table default.table_49a9f7854b4c4847bce27e16296322b5 [id=cf4da31cb40447199dc079b8b6e709bc]
I20260807 19:42:10.533181 7782 tablet_loader.cc:96] loaded metadata for tablet 0d47a7a1285948468c0e6e1bddcd26d4 (table default.table_c4966a160ae2456ebe5e58c41a9ac129 [id=490e9d5bbd5347aaae98c3a81cb3fe1e])
I20260807 19:42:10.534011 7782 tablet_loader.cc:96] loaded metadata for tablet 248c1670db7244cfac99dd4817b513a4 (table default.table_472c6a33c87c4f1b9cdb926a2e5a6344 [id=0b2f82ef918b47d59b34a5b5757b9486])
I20260807 19:42:10.534438 7782 tablet_loader.cc:96] loaded metadata for tablet 769aecb6d5ca42e09edba0277a022901 (table default.table_5d5e7f4976094d7a942e0ba75696c8f7 [id=9918b1d3805d424b92055cf267b8a4ab])
I20260807 19:42:10.534890 7782 tablet_loader.cc:96] loaded metadata for tablet 807180c74470485182a926ab39d79ab4 (table default.table_ca953b708a5a408898f72c483fc31ecf [id=bff436a0a60542b9b0fecacf4a6ed1b6])
I20260807 19:42:10.535321 7782 tablet_loader.cc:96] loaded metadata for tablet 9a45d0765a2c4d93ac9448496ce9c325 (table default.table_49a9f7854b4c4847bce27e16296322b5 [id=cf4da31cb40447199dc079b8b6e709bc])
I20260807 19:42:10.535969 7782 tablet_loader.cc:96] loaded metadata for tablet d328d2e772ed4a6bb8597658f907f0f0 (table default.table_271147f124bf45a88d332b72ac8d3e13 [id=a75161def1a34954bf0846ab30a8cc38])
I20260807 19:42:10.536626 7782 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:10.538076 7782 catalog_manager.cc:1259] Loaded cluster ID: a80f7b000fe149f3a73e545a28c78fda
I20260807 19:42:10.538223 7782 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:10.540268 7782 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:10.541119 7399 heartbeater.cc:347] Connected to a master server at 127.6.109.61:39263
I20260807 19:42:10.543583 7710 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" instance_seqno: 1786131727626128) as {username='slave'} at 127.6.109.2:55059; Asking this server to re-register.
I20260807 19:42:10.544060 7646 consensus_queue.cc:1035] T d328d2e772ed4a6bb8597658f907f0f0 P 051d464d644a4d4486462dfb18274c6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:10.544924 7782 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054: Loaded TSK: 0
I20260807 19:42:10.545397 7399 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:10.546962 7782 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:42:10.548413 7782 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260807 19:42:10.548431 7710 ts_manager.cc:194] Registered new tserver with Master: 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483)
I20260807 19:42:10.550020 7782 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260807 19:42:10.573000 7787 consensus_queue.cc:1035] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:10.578084 6855 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46550:
name: "default.Table_8461316ca98b4afab4331f54eb8d5600"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:10.578676 6854 master_service.cc:946] ReplaceTablet: received request to replace tablet 0d47a7a1285948468c0e6e1bddcd26d4 from {username='slave'} at 127.0.0.1:46550
I20260807 19:42:10.579114 6852 master_service.cc:946] ReplaceTablet: received request to replace tablet d328d2e772ed4a6bb8597658f907f0f0 from {username='slave'} at 127.0.0.1:46550
I20260807 19:42:10.579159 6853 master_service.cc:946] ReplaceTablet: received request to replace tablet 0d47a7a1285948468c0e6e1bddcd26d4 from {username='slave'} at 127.0.0.1:46550
I20260807 19:42:10.579510 6850 master_service.cc:946] ReplaceTablet: received request to replace tablet 9a45d0765a2c4d93ac9448496ce9c325 from {username='slave'} at 127.0.0.1:46550
I20260807 19:42:10.579566 6851 master_service.cc:946] ReplaceTablet: received request to replace tablet 248c1670db7244cfac99dd4817b513a4 from {username='slave'} at 127.0.0.1:46550
I20260807 19:42:10.580278 6856 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46550:
table { table_name: "default.Table_ca953b708a5a408898f72c483fc31ecf" } new_table_name: "default.Table_eb8c970c75a34df9a37b1812e7f5cc76" modify_external_catalogs: true
I20260807 19:42:10.580333 6858 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46550:
name: "default.Table_8ea588612f6c47c79ac5c31aa592809e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:10.580889 6857 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46550:
table { table_name: "default.Table_5d5e7f4976094d7a942e0ba75696c8f7" } modify_external_catalogs: true
I20260807 19:42:10.581125 7256 heartbeater.cc:503] Master 127.6.109.62:43417 was elected leader, sending a full tablet report...
I20260807 19:42:10.581260 6857 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46550:
table { table_name: "default.Table_5d5e7f4976094d7a942e0ba75696c8f7" } modify_external_catalogs: true
I20260807 19:42:10.584846 7663 consensus_queue.cc:1035] T d328d2e772ed4a6bb8597658f907f0f0 P 051d464d644a4d4486462dfb18274c6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:10.578032 6859 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46550:
name: "default.Table_24957fcd29dc4a3db1b0ab2dcf131779"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260807 19:42:10.604633 7078 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_c4966a160ae2456ebe5e58c41a9ac129
*** Check failure stack trace: ***
@ 0x7f64891cf9ad google::LogMessage::Fail() at ??:0
@ 0x7f64891d288a google::LogMessage::SendToLog() at ??:0
@ 0x7f64891cf4de google::LogMessage::Flush() at ??:0
@ 0x7f64891cf6d1 google::LogMessage::~LogMessage() at ??:0
I20260807 19:42:10.613421 7548 heartbeater.cc:347] Connected to a master server at 127.6.109.61:39263
I20260807 19:42:10.613701 7548 heartbeater.cc:503] Master 127.6.109.61:39263 was elected leader, sending a full tablet report...
@ 0x7f649ed6dba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7f649ed6c901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
W20260807 19:42:10.627290 6846 consensus_peers.cc:487] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 -> Peer 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129): Couldn't send request to peer 7b31edda0c4a48489ad60fc59c44ed31. Status: Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 19:42:10.627470 7720 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 6 FOLLOWER]: Refusing update from remote peer 959ac7807a8b42a59c61a23d4605c054: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 27. (index mismatch)
I20260807 19:42:10.628643 7781 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 6, Time since last communication: 0.000s
@ 0x7f649ed7366c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7f649ed734e9 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f648bdf9eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f648f0d36db start_thread at ??:0
@ 0x7f648806961f clone at ??:0
I20260807 19:42:10.691475 7751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 959ac7807a8b42a59c61a23d4605c054. Latest consensus state: current_term: 6 leader_uuid: "959ac7807a8b42a59c61a23d4605c054" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:10.691862 7751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:10.692706 7710 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "bafdff0fd60146539631a476c4257e16" instance_seqno: 1786131728431251) as {username='slave'} at 127.6.109.3:52831; Asking this server to re-register.
I20260807 19:42:10.693085 7802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 6 leader_uuid: "959ac7807a8b42a59c61a23d4605c054" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:10.693500 7802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:10.693928 7548 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:10.697546 7710 ts_manager.cc:194] Registered new tserver with Master: bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
I20260807 19:42:10.710409 7798 mvcc.cc:205] Tried to move back new op lower bound from 7315995565964845056 to 7315995561306697728. Current Snapshot: MvccSnapshot[applied={T|T < 7315995565964845056 or (T in {7315995565964845056})}]
W20260807 19:42:10.877189 7696 connection.cc:535] server connection from 127.0.0.1:41926 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 19:42:10.927772 7657 consensus_queue.cc:1035] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da [LEADER]: Connected to new peer: Peer: permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:10.936268 7657 consensus_queue.cc:1035] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da [LEADER]: Connected to new peer: Peer: permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:11.134675 7802 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:42:11.134972 7802 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:11.136307 7646 consensus_queue.cc:1035] T 0d47a7a1285948468c0e6e1bddcd26d4 P 051d464d644a4d4486462dfb18274c6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:11.137145 7802 leader_election.cc:288] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129)
W20260807 19:42:11.138187 7696 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
W20260807 19:42:11.141479 7697 leader_election.cc:334] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129): Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: connect: Connection refused (error 111)
W20260807 19:42:11.144618 7696 leader_election.cc:334] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
I20260807 19:42:11.144794 7696 leader_election.cc:302] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b782e30a8d44e7b94f080aa6327134f; no voters: 7b31edda0c4a48489ad60fc59c44ed31, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:42:11.145241 7802 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:42:11.145795 7646 consensus_queue.cc:1035] T 0d47a7a1285948468c0e6e1bddcd26d4 P 051d464d644a4d4486462dfb18274c6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:11.258630 7646 consensus_queue.cc:1035] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:11.266206 7829 consensus_queue.cc:1035] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:11.290081 7829 consensus_queue.cc:1035] T 769aecb6d5ca42e09edba0277a022901 P 051d464d644a4d4486462dfb18274c6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:11.299736 7832 consensus_queue.cc:1035] T 769aecb6d5ca42e09edba0277a022901 P 051d464d644a4d4486462dfb18274c6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:11.830149 7838 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:42:11.830379 7838 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:11.832053 7838 leader_election.cc:288] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129)
W20260807 19:42:11.834429 7697 leader_election.cc:334] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129): Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: connect: Connection refused (error 111)
W20260807 19:42:11.834910 7696 leader_election.cc:334] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
I20260807 19:42:11.835083 7696 leader_election.cc:302] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b782e30a8d44e7b94f080aa6327134f; no voters: 7b31edda0c4a48489ad60fc59c44ed31, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:42:11.835490 7838 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:42:12.959522 7861 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:42:12.959779 7861 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:12.960491 7861 leader_election.cc:288] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129)
W20260807 19:42:12.962810 7697 leader_election.cc:334] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129): Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: connect: Connection refused (error 111)
W20260807 19:42:12.963017 7696 leader_election.cc:334] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
I20260807 19:42:12.963172 7696 leader_election.cc:302] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b782e30a8d44e7b94f080aa6327134f; no voters: 7b31edda0c4a48489ad60fc59c44ed31, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:42:12.963438 7861 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:42:13.606285 6580 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskeMwJpl/build/asan/bin/kudu
/tmp/dist-test-taskeMwJpl/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.109.60:43129
--webserver_interface=127.6.109.60
--webserver_port=36383
--builtin_ntp_servers=127.6.109.20:40195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129
--hive_metastore_uris=thrift://127.0.0.1:38987
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260807 19:42:13.942118 7880 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:42:13.942368 7880 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:13.943249 7880 leader_election.cc:288] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129)
W20260807 19:42:13.954888 7696 leader_election.cc:334] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
W20260807 19:42:13.955214 7697 leader_election.cc:334] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129): Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: connect: Connection refused (error 111)
I20260807 19:42:13.955436 7697 leader_election.cc:302] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b782e30a8d44e7b94f080aa6327134f; no voters: 7b31edda0c4a48489ad60fc59c44ed31, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:42:13.955775 7880 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260807 19:42:14.008253 7873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:14.008589 7873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:14.008669 7873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:14.016794 7873 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:14.016930 7873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:14.016978 7873 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:14.017033 7873 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:14.026906 7873 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.109.20:40195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:38987
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.109.60:43129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.109.60
--webserver_port=36383
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:14.028307 7873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:14.030333 7873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:14.043995 7889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:14.044075 7888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:14.044051 7892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:14.045125 7873 server_base.cc:1034] running on GCE node
I20260807 19:42:14.046100 7873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:14.047719 7873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:14.048995 7873 hybrid_clock.cc:648] HybridClock initialized: now 1786131734048927 us; error 35 us; skew 500 ppm
I20260807 19:42:14.049351 7873 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:42:14.051904 7873 webserver.cc:460] Webserver started at http://127.6.109.60:36383/ using document root <none> and password file <none>
I20260807 19:42:14.052639 7873 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:14.052805 7873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:14.058494 7873 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:42:14.061606 7898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:14.063320 7873 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 19:42:14.063535 7873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data,/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/wal
uuid: "7b31edda0c4a48489ad60fc59c44ed31"
format_stamp: "Formatted at 2026-08-07 19:42:05 on dist-test-slave-p3t5"
I20260807 19:42:14.065016 7873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:14.084589 7873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:14.085548 7873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:14.085940 7873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:14.105731 7873 rpc_server.cc:307] RPC server started. Bound to: 127.6.109.60:43129
I20260807 19:42:14.105753 7951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.109.60:43129 every 8 connection(s)
I20260807 19:42:14.107478 7873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data/info.pb
I20260807 19:42:14.110109 6580 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskeMwJpl/build/asan/bin/kudu as pid 7873
I20260807 19:42:14.115929 7952 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:42:14.119786 7952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: Bootstrap starting.
I20260807 19:42:14.131214 7952 log.cc:824] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:14.157270 7952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: Bootstrap replayed 1/1 log segments. Stats: ops{read=26 overwritten=1 applied=25 ignored=0} inserts{seen=17 ignored=0} mutations{seen=13 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:42:14.157948 7952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: Bootstrap complete.
I20260807 19:42:14.165593 7952 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 5 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:14.166713 7952 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 5 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b31edda0c4a48489ad60fc59c44ed31, State: Initialized, Role: FOLLOWER
I20260807 19:42:14.167428 7952 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 25, Last appended: 5.25, Last appended by leader: 25, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:14.168390 7956 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 5 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:14.168763 7956 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:14.169687 7952 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:14.185511 7971 catalog_manager.cc:1259] Loaded cluster ID: a80f7b000fe149f3a73e545a28c78fda
I20260807 19:42:14.185756 7971 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: loading cluster ID for follower catalog manager: success
I20260807 19:42:14.190138 7971 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: acquiring CA information for follower catalog manager: success
I20260807 19:42:14.193235 7971 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 19:42:14.198746 7873 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:42:14.208202 7873 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:42:14.217722 7255 heartbeater.cc:347] Connected to a master server at 127.6.109.60:43129
I20260807 19:42:14.221859 7915 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "051d464d644a4d4486462dfb18274c6a" instance_seqno: 1786131726838357) as {username='slave'} at 127.6.109.1:56155; Asking this server to re-register.
I20260807 19:42:14.222841 7255 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:14.224786 7915 ts_manager.cc:194] Registered new tserver with Master: 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467)
I20260807 19:42:14.316807 7546 heartbeater.cc:347] Connected to a master server at 127.6.109.60:43129
I20260807 19:42:14.319698 7914 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "bafdff0fd60146539631a476c4257e16" instance_seqno: 1786131728431251) as {username='slave'} at 127.6.109.3:53913; Asking this server to re-register.
I20260807 19:42:14.320590 7546 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:14.322423 7914 ts_manager.cc:194] Registered new tserver with Master: bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
I20260807 19:42:14.430200 7397 heartbeater.cc:347] Connected to a master server at 127.6.109.60:43129
I20260807 19:42:14.432129 7914 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" instance_seqno: 1786131727626128) as {username='slave'} at 127.6.109.2:54753; Asking this server to re-register.
I20260807 19:42:14.432811 7397 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:14.434406 7914 ts_manager.cc:194] Registered new tserver with Master: 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483)
I20260807 19:42:14.611918 7956 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:14.612314 7956 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:14.614403 7956 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:42:14.615103 7901 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
W20260807 19:42:14.617213 7901 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
I20260807 19:42:14.622488 7720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7b31edda0c4a48489ad60fc59c44ed31" candidate_term: 6 candidate_status { last_received { term: 5 index: 25 } } ignore_live_leader: false dest_uuid: "1b782e30a8d44e7b94f080aa6327134f" is_pre_election: true
I20260807 19:42:14.622817 7720 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7b31edda0c4a48489ad60fc59c44ed31 in current term 6: Already voted for candidate 959ac7807a8b42a59c61a23d4605c054 in this term.
I20260807 19:42:14.623617 7902 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:42:14.624127 7956 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:42:14.628543 7956 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:42:15.207953 7989 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:15.208190 7989 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:15.209019 7989 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
I20260807 19:42:15.209694 7720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7b31edda0c4a48489ad60fc59c44ed31" candidate_term: 7 candidate_status { last_received { term: 5 index: 25 } } ignore_live_leader: false dest_uuid: "1b782e30a8d44e7b94f080aa6327134f" is_pre_election: true
I20260807 19:42:15.210072 7720 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7b31edda0c4a48489ad60fc59c44ed31 for term 7 because replica has last-logged OpId of term: 6 index: 28, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 25.
W20260807 19:42:15.211354 7901 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
I20260807 19:42:15.211479 7901 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:42:15.211771 7989 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:42:15.320116 7996 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:42:15.320361 7996 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:15.321324 7996 leader_election.cc:288] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129)
W20260807 19:42:15.323706 7696 leader_election.cc:334] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
I20260807 19:42:15.329226 7927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1b782e30a8d44e7b94f080aa6327134f" candidate_term: 7 candidate_status { last_received { term: 6 index: 28 } } ignore_live_leader: false dest_uuid: "7b31edda0c4a48489ad60fc59c44ed31" is_pre_election: true
I20260807 19:42:15.329797 7927 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b782e30a8d44e7b94f080aa6327134f in term 6.
I20260807 19:42:15.330363 7697 leader_election.cc:302] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b782e30a8d44e7b94f080aa6327134f, 7b31edda0c4a48489ad60fc59c44ed31; no voters: 959ac7807a8b42a59c61a23d4605c054
I20260807 19:42:15.330725 7996 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:42:15.330874 7996 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:42:15.330984 7996 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:42:15.333164 7996 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:15.333899 7996 leader_election.cc:288] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [CANDIDATE]: Term 7 election: Requested vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129)
I20260807 19:42:15.334563 7927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1b782e30a8d44e7b94f080aa6327134f" candidate_term: 7 candidate_status { last_received { term: 6 index: 28 } } ignore_live_leader: false dest_uuid: "7b31edda0c4a48489ad60fc59c44ed31"
I20260807 19:42:15.334780 7927 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 6 FOLLOWER]: Advancing to term 7
W20260807 19:42:15.335424 7696 leader_election.cc:334] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
I20260807 19:42:15.337360 7927 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b782e30a8d44e7b94f080aa6327134f in term 7.
I20260807 19:42:15.337916 7697 leader_election.cc:302] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b782e30a8d44e7b94f080aa6327134f, 7b31edda0c4a48489ad60fc59c44ed31; no voters: 959ac7807a8b42a59c61a23d4605c054
I20260807 19:42:15.338267 7996 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 7 FOLLOWER]: Leader election won for term 7
I20260807 19:42:15.338732 7996 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [term 7 LEADER]: Becoming Leader. State: Replica: 1b782e30a8d44e7b94f080aa6327134f, State: Running, Role: LEADER
I20260807 19:42:15.339208 7996 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 6.28, Last appended by leader: 28, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:15.340842 8001 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b782e30a8d44e7b94f080aa6327134f. Latest consensus state: current_term: 7 leader_uuid: "1b782e30a8d44e7b94f080aa6327134f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:15.341177 8001 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:15.482563 7640 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:42:15.482795 7640 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:42:15.862419 7927 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 7 FOLLOWER]: Refusing update from remote peer 1b782e30a8d44e7b94f080aa6327134f: Log matching property violated. Preceding OpId in replica: term: 5 index: 25. Preceding OpId from leader: term: 7 index: 29. (index mismatch)
I20260807 19:42:15.863451 8007 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 25, Time since last communication: 0.000s
W20260807 19:42:15.865272 7696 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f -> Peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Couldn't send request to peer 959ac7807a8b42a59c61a23d4605c054. Status: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 19:42:15.870338 8009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b782e30a8d44e7b94f080aa6327134f. Latest consensus state: current_term: 7 leader_uuid: "1b782e30a8d44e7b94f080aa6327134f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:15.870745 8009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:15.884186 8007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "1b782e30a8d44e7b94f080aa6327134f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:15.884584 8007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:15.885563 8007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 7 leader_uuid: "1b782e30a8d44e7b94f080aa6327134f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:15.885926 8007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:15.886641 8009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "1b782e30a8d44e7b94f080aa6327134f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:15.886948 8009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:15.886583 8018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "1b782e30a8d44e7b94f080aa6327134f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:15.887070 8018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:15.986970 8003 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:15.990700 8003 catalog_manager.cc:669] Loaded metadata for table default.table_472c6a33c87c4f1b9cdb926a2e5a6344 [id=0b2f82ef918b47d59b34a5b5757b9486]
I20260807 19:42:15.991240 8003 catalog_manager.cc:669] Loaded metadata for table default.table_c4966a160ae2456ebe5e58c41a9ac129 [id=490e9d5bbd5347aaae98c3a81cb3fe1e]
I20260807 19:42:15.991644 8003 catalog_manager.cc:669] Loaded metadata for table default.table_5d5e7f4976094d7a942e0ba75696c8f7 [id=9918b1d3805d424b92055cf267b8a4ab]
I20260807 19:42:15.992069 8003 catalog_manager.cc:669] Loaded metadata for table default.table_271147f124bf45a88d332b72ac8d3e13 [id=a75161def1a34954bf0846ab30a8cc38]
I20260807 19:42:15.992445 8003 catalog_manager.cc:669] Loaded metadata for table default.table_ca953b708a5a408898f72c483fc31ecf [id=bff436a0a60542b9b0fecacf4a6ed1b6]
I20260807 19:42:15.992863 8003 catalog_manager.cc:669] Loaded metadata for table default.table_49a9f7854b4c4847bce27e16296322b5 [id=cf4da31cb40447199dc079b8b6e709bc]
I20260807 19:42:15.995167 8003 tablet_loader.cc:96] loaded metadata for tablet 248c1670db7244cfac99dd4817b513a4 (table default.table_472c6a33c87c4f1b9cdb926a2e5a6344 [id=0b2f82ef918b47d59b34a5b5757b9486])
I20260807 19:42:15.995507 8003 tablet_loader.cc:96] loaded metadata for tablet 541fb835769b4d1f92407b288847cdee (table default.table_c4966a160ae2456ebe5e58c41a9ac129 [id=490e9d5bbd5347aaae98c3a81cb3fe1e])
I20260807 19:42:15.995841 8003 tablet_loader.cc:96] loaded metadata for tablet 769aecb6d5ca42e09edba0277a022901 (table default.table_5d5e7f4976094d7a942e0ba75696c8f7 [id=9918b1d3805d424b92055cf267b8a4ab])
I20260807 19:42:15.996192 8003 tablet_loader.cc:96] loaded metadata for tablet 807180c74470485182a926ab39d79ab4 (table default.table_ca953b708a5a408898f72c483fc31ecf [id=bff436a0a60542b9b0fecacf4a6ed1b6])
I20260807 19:42:15.996476 8003 tablet_loader.cc:96] loaded metadata for tablet 827e9a16eeb24a2cadf42fbb0b12a872 (table default.table_271147f124bf45a88d332b72ac8d3e13 [id=a75161def1a34954bf0846ab30a8cc38])
I20260807 19:42:15.996797 8003 tablet_loader.cc:96] loaded metadata for tablet 9a45d0765a2c4d93ac9448496ce9c325 (table default.table_49a9f7854b4c4847bce27e16296322b5 [id=cf4da31cb40447199dc079b8b6e709bc])
I20260807 19:42:15.997557 8003 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:15.998778 8003 catalog_manager.cc:1259] Loaded cluster ID: a80f7b000fe149f3a73e545a28c78fda
I20260807 19:42:15.998867 8003 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:16.000340 8003 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:16.001401 8003 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1b782e30a8d44e7b94f080aa6327134f: Loaded TSK: 0
I20260807 19:42:16.002591 8003 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:42:16.003839 8003 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260807 19:42:16.004914 8003 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260807 19:42:16.102010 7709 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41916:
table { table_name: "default.Table_49a9f7854b4c4847bce27e16296322b5" } new_table_name: "default.Table_e60fcc04a04e4a0d9456e55b94757d74" modify_external_catalogs: true
I20260807 19:42:16.102070 7710 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41916:
table { table_name: "default.Table_271147f124bf45a88d332b72ac8d3e13" } new_table_name: "default.Table_830c2e1165884867b9cb12c80f090b48" modify_external_catalogs: true
I20260807 19:42:16.102875 7703 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41916:
table { table_name: "default.Table_ca953b708a5a408898f72c483fc31ecf" } new_table_name: "default.Table_eb8c970c75a34df9a37b1812e7f5cc76" modify_external_catalogs: true
I20260807 19:42:16.102792 7706 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41916:
name: "default.Table_1455c4f504ff4603bc11da85f16cc69f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:16.103231 7708 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41916:
table { table_name: "default.Table_c4966a160ae2456ebe5e58c41a9ac129" } new_table_name: "default.Table_d6db13f17872439a8858b3d5fd56bfde" modify_external_catalogs: true
I20260807 19:42:16.103168 7704 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41916:
name: "default.Table_8461316ca98b4afab4331f54eb8d5600"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:16.103714 7701 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41916:
name: "default.Table_8ea588612f6c47c79ac5c31aa592809e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:16.104429 7705 master_service.cc:946] ReplaceTablet: received request to replace tablet 0d47a7a1285948468c0e6e1bddcd26d4 from {username='slave'} at 127.0.0.1:41916
I20260807 19:42:16.104061 7702 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41916:
table { table_name: "default.Table_5d5e7f4976094d7a942e0ba75696c8f7" } modify_external_catalogs: true
I20260807 19:42:16.104990 7702 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41916:
table { table_name: "default.Table_5d5e7f4976094d7a942e0ba75696c8f7" } modify_external_catalogs: true
I20260807 19:42:16.103901 7707 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41916:
name: "default.Table_61514c4f10b54148a57bbaf295205ace"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:16.106000 7705 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41916:
name: "default.Table_24957fcd29dc4a3db1b0ab2dcf131779"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260807 19:42:16.111478 7766 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_c4966a160ae2456ebe5e58c41a9ac129
*** Check failure stack trace: ***
@ 0x7f17b8cd19ad google::LogMessage::Fail() at ??:0
@ 0x7f17b8cd488a google::LogMessage::SendToLog() at ??:0
@ 0x7f17b8cd14de google::LogMessage::Flush() at ??:0
@ 0x7f17b8cd16d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7f17ce86fba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7f17ce86e901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f17ce87566c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7f17ce8754e9 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f17bb8fbeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f17bebd56db start_thread at ??:0
@ 0x7f17b7b6b61f clone at ??:0
W20260807 19:42:16.331578 7436 connection.cc:535] client connection to 127.6.109.61:39263 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 19:42:16.331624 7140 connection.cc:535] client connection to 127.6.109.61:39263 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 19:42:16.331765 7559 connection.cc:535] client connection to 127.6.109.61:39263 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 19:42:16.332573 7559 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.6.109.61:39263)
W20260807 19:42:16.332922 7559 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.6.109.61:39263)
W20260807 19:42:16.333451 7559 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.6.109.61:39263)
W20260807 19:42:16.334335 7559 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.6.109.61:39263)
I20260807 19:42:16.373943 8018 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1b782e30a8d44e7b94f080aa6327134f)
I20260807 19:42:16.374158 8018 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:16.374843 8018 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:42:16.377581 7901 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
W20260807 19:42:16.377840 7902 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:42:16.378013 7902 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:42:16.378278 8018 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:42:16.914299 6580 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskeMwJpl/build/asan/bin/kudu
/tmp/dist-test-taskeMwJpl/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.109.62:43417
--webserver_interface=127.6.109.62
--webserver_port=37311
--builtin_ntp_servers=127.6.109.20:40195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129
--hive_metastore_uris=thrift://127.0.0.1:38987
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260807 19:42:17.021044 8045 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1b782e30a8d44e7b94f080aa6327134f)
I20260807 19:42:17.021276 8045 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:17.025537 8045 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:42:17.026983 7902 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:42:17.027210 7901 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
I20260807 19:42:17.027418 7901 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:42:17.027719 8045 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260807 19:42:17.280836 8044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:17.281244 8044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:17.281397 8044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:17.289539 8044 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:17.289731 8044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:17.289832 8044 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:17.289920 8044 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:17.299865 8044 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.109.20:40195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:38987
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.109.62:43417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.109.62
--webserver_port=37311
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:17.301419 8044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:17.303457 8044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:17.316632 8058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:17.316601 8056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:17.316601 8055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:17.317916 8044 server_base.cc:1034] running on GCE node
I20260807 19:42:17.318962 8044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:17.320647 8044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:17.321907 8044 hybrid_clock.cc:648] HybridClock initialized: now 1786131737321821 us; error 49 us; skew 500 ppm
I20260807 19:42:17.322288 8044 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:42:17.330475 8044 webserver.cc:460] Webserver started at http://127.6.109.62:37311/ using document root <none> and password file <none>
I20260807 19:42:17.331314 8044 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:17.331493 8044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:17.338178 8044 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:42:17.341790 8065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:17.343333 8044 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:42:17.343575 8044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/data,/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/wal
uuid: "959ac7807a8b42a59c61a23d4605c054"
format_stamp: "Formatted at 2026-08-07 19:42:04 on dist-test-slave-p3t5"
I20260807 19:42:17.344681 8044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:17.363741 8044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:17.364656 8044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:17.364934 8044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:17.384867 8044 rpc_server.cc:307] RPC server started. Bound to: 127.6.109.62:43417
I20260807 19:42:17.384909 8117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.109.62:43417 every 8 connection(s)
I20260807 19:42:17.386746 8044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/data/info.pb
I20260807 19:42:17.394187 8118 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:42:17.397125 6580 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskeMwJpl/build/asan/bin/kudu as pid 8044
I20260807 19:42:17.398119 8118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054: Bootstrap starting.
I20260807 19:42:17.413936 8118 log.cc:824] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:17.439920 7398 heartbeater.cc:347] Connected to a master server at 127.6.109.62:43417
I20260807 19:42:17.443032 8118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054: Bootstrap replayed 1/1 log segments. Stats: ops{read=30 overwritten=0 applied=25 ignored=0} inserts{seen=17 ignored=0} mutations{seen=13 ignored=0} orphaned_commits=0. Pending: 5 replicates
I20260807 19:42:17.443724 8118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054: Bootstrap complete.
I20260807 19:42:17.452765 8118 raft_consensus.cc:348] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 6 FOLLOWER]: Replica starting. Triggering 5 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:17.456969 8118 raft_consensus.cc:729] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 6 FOLLOWER]: Becoming Follower/Learner. State: Replica: 959ac7807a8b42a59c61a23d4605c054, State: Initialized, Role: FOLLOWER
I20260807 19:42:17.457765 8118 consensus_queue.cc:260] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 25, Last appended: 6.30, Last appended by leader: 30, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:17.458782 8126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 6 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:17.459221 8126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:17.459903 8118 sys_catalog.cc:564] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:17.478268 8141 catalog_manager.cc:1259] Loaded cluster ID: a80f7b000fe149f3a73e545a28c78fda
I20260807 19:42:17.478511 8141 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054: loading cluster ID for follower catalog manager: success
I20260807 19:42:17.483484 8141 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054: acquiring CA information for follower catalog manager: success
I20260807 19:42:17.486827 8141 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 19:42:17.488317 8044 master_runner.cc:186] Error getting master registration for 127.6.109.61:39263: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111),
I20260807 19:42:17.500607 8044 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:42:17.727563 8148 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1b782e30a8d44e7b94f080aa6327134f)
I20260807 19:42:17.727895 7256 heartbeater.cc:347] Connected to a master server at 127.6.109.62:43417
I20260807 19:42:17.728133 7256 heartbeater.cc:503] Master 127.6.109.62:43417 was elected leader, sending a full tablet report...
I20260807 19:42:17.727948 8148 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:17.728767 8148 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
I20260807 19:42:17.732441 8083 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "051d464d644a4d4486462dfb18274c6a" instance_seqno: 1786131726838357) as {username='slave'} at 127.6.109.1:39069; Asking this server to re-register.
I20260807 19:42:17.733834 7256 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:17.736034 8083 ts_manager.cc:194] Registered new tserver with Master: 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467)
W20260807 19:42:17.737757 7902 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:42:17.739900 8093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7b31edda0c4a48489ad60fc59c44ed31" candidate_term: 8 candidate_status { last_received { term: 7 index: 29 } } ignore_live_leader: false dest_uuid: "959ac7807a8b42a59c61a23d4605c054" is_pre_election: true
I20260807 19:42:17.740559 8093 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b31edda0c4a48489ad60fc59c44ed31 in term 6.
I20260807 19:42:17.741204 7901 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31, 959ac7807a8b42a59c61a23d4605c054; no voters: 1b782e30a8d44e7b94f080aa6327134f
I20260807 19:42:17.741582 8148 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 19:42:17.741777 8148 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1b782e30a8d44e7b94f080aa6327134f)
I20260807 19:42:17.741955 8148 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:42:17.745227 8148 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:17.746258 8148 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 8 election: Requested vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
I20260807 19:42:17.746774 8093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7b31edda0c4a48489ad60fc59c44ed31" candidate_term: 8 candidate_status { last_received { term: 7 index: 29 } } ignore_live_leader: false dest_uuid: "959ac7807a8b42a59c61a23d4605c054"
I20260807 19:42:17.747032 8093 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 6 FOLLOWER]: Advancing to term 8
W20260807 19:42:17.748188 7902 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:42:17.752087 8093 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b31edda0c4a48489ad60fc59c44ed31 in term 8.
I20260807 19:42:17.752629 7901 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31, 959ac7807a8b42a59c61a23d4605c054; no voters: 1b782e30a8d44e7b94f080aa6327134f
I20260807 19:42:17.753053 8148 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 8 FOLLOWER]: Leader election won for term 8
I20260807 19:42:17.753619 8148 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 8 LEADER]: Becoming Leader. State: Replica: 7b31edda0c4a48489ad60fc59c44ed31, State: Running, Role: LEADER
I20260807 19:42:17.754122 8148 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 7.29, Last appended by leader: 29, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:17.755992 8153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b31edda0c4a48489ad60fc59c44ed31. Latest consensus state: current_term: 8 leader_uuid: "7b31edda0c4a48489ad60fc59c44ed31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:17.756367 8153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:17.757189 8155 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:17.759802 8155 catalog_manager.cc:669] Loaded metadata for table default.table_472c6a33c87c4f1b9cdb926a2e5a6344 [id=0b2f82ef918b47d59b34a5b5757b9486]
I20260807 19:42:17.760339 8155 catalog_manager.cc:669] Loaded metadata for table default.table_c4966a160ae2456ebe5e58c41a9ac129 [id=490e9d5bbd5347aaae98c3a81cb3fe1e]
I20260807 19:42:17.760739 8155 catalog_manager.cc:669] Loaded metadata for table default.table_5d5e7f4976094d7a942e0ba75696c8f7 [id=9918b1d3805d424b92055cf267b8a4ab]
I20260807 19:42:17.761112 8155 catalog_manager.cc:669] Loaded metadata for table default.table_271147f124bf45a88d332b72ac8d3e13 [id=a75161def1a34954bf0846ab30a8cc38]
I20260807 19:42:17.761601 8155 catalog_manager.cc:669] Loaded metadata for table default.table_ca953b708a5a408898f72c483fc31ecf [id=bff436a0a60542b9b0fecacf4a6ed1b6]
I20260807 19:42:17.762004 8155 catalog_manager.cc:669] Loaded metadata for table default.table_49a9f7854b4c4847bce27e16296322b5 [id=cf4da31cb40447199dc079b8b6e709bc]
I20260807 19:42:17.765038 8155 tablet_loader.cc:96] loaded metadata for tablet 248c1670db7244cfac99dd4817b513a4 (table default.table_472c6a33c87c4f1b9cdb926a2e5a6344 [id=0b2f82ef918b47d59b34a5b5757b9486])
I20260807 19:42:17.765389 8155 tablet_loader.cc:96] loaded metadata for tablet 541fb835769b4d1f92407b288847cdee (table default.table_c4966a160ae2456ebe5e58c41a9ac129 [id=490e9d5bbd5347aaae98c3a81cb3fe1e])
I20260807 19:42:17.765795 8155 tablet_loader.cc:96] loaded metadata for tablet 769aecb6d5ca42e09edba0277a022901 (table default.table_5d5e7f4976094d7a942e0ba75696c8f7 [id=9918b1d3805d424b92055cf267b8a4ab])
I20260807 19:42:17.766188 8155 tablet_loader.cc:96] loaded metadata for tablet 807180c74470485182a926ab39d79ab4 (table default.table_ca953b708a5a408898f72c483fc31ecf [id=bff436a0a60542b9b0fecacf4a6ed1b6])
I20260807 19:42:17.766487 8155 tablet_loader.cc:96] loaded metadata for tablet 827e9a16eeb24a2cadf42fbb0b12a872 (table default.table_271147f124bf45a88d332b72ac8d3e13 [id=a75161def1a34954bf0846ab30a8cc38])
I20260807 19:42:17.766872 8155 tablet_loader.cc:96] loaded metadata for tablet 9a45d0765a2c4d93ac9448496ce9c325 (table default.table_49a9f7854b4c4847bce27e16296322b5 [id=cf4da31cb40447199dc079b8b6e709bc])
I20260807 19:42:17.767622 8155 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:17.768831 8155 catalog_manager.cc:1259] Loaded cluster ID: a80f7b000fe149f3a73e545a28c78fda
I20260807 19:42:17.768970 8155 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:17.770546 8155 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:17.771556 8155 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: Loaded TSK: 0
I20260807 19:42:17.772596 8155 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:42:17.773654 8155 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260807 19:42:17.774526 8155 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260807 19:42:17.827260 7913 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38290:
name: "default.Table_8cc3d0e85b19426f8df95f1894914926"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:17.827672 7912 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38290:
name: "default.Table_5d300b1b9ec343b88bbb7481707c963a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:17.828342 7915 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38290:
name: "default.Table_b6eca8c0167f4d80b6ef4f8125aea68f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:17.827302 7911 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38290:
table { table_name: "default.Table_472c6a33c87c4f1b9cdb926a2e5a6344" } new_table_name: "default.Table_cda45f853cca4f52a3371bbaec9c7750" modify_external_catalogs: true
I20260807 19:42:17.830294 7906 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38290:
table { table_name: "default.Table_271147f124bf45a88d332b72ac8d3e13" } new_table_name: "default.Table_830c2e1165884867b9cb12c80f090b48" modify_external_catalogs: true
I20260807 19:42:17.831311 7547 heartbeater.cc:347] Connected to a master server at 127.6.109.62:43417
I20260807 19:42:17.831231 7907 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38290:
table { table_name: "default.Table_c4966a160ae2456ebe5e58c41a9ac129" } new_table_name: "default.Table_d6db13f17872439a8858b3d5fd56bfde" modify_external_catalogs: true
I20260807 19:42:17.831730 7908 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38290:
name: "default.Table_61514c4f10b54148a57bbaf295205ace"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:17.832288 7909 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38290:
name: "default.Table_1455c4f504ff4603bc11da85f16cc69f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:17.832729 7910 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38290:
name: "default.Table_f2d1ca595d0c43ec982c1377f39155ee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:17.833847 8083 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "bafdff0fd60146539631a476c4257e16" instance_seqno: 1786131728431251) as {username='slave'} at 127.6.109.3:39537; Asking this server to re-register.
I20260807 19:42:17.831254 7914 master_service.cc:946] ReplaceTablet: received request to replace tablet 0d47a7a1285948468c0e6e1bddcd26d4 from {username='slave'} at 127.0.0.1:38290
I20260807 19:42:17.834764 7547 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:17.835232 7914 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38290:
table { table_name: "default.Table_49a9f7854b4c4847bce27e16296322b5" } new_table_name: "default.Table_e60fcc04a04e4a0d9456e55b94757d74" modify_external_catalogs: true
I20260807 19:42:17.836694 8083 ts_manager.cc:194] Registered new tserver with Master: bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
F20260807 19:42:17.838042 7970 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_c4966a160ae2456ebe5e58c41a9ac129
*** Check failure stack trace: ***
@ 0x7f83f6c929ad google::LogMessage::Fail() at ??:0
@ 0x7f83f6c9588a google::LogMessage::SendToLog() at ??:0
@ 0x7f83f6c924de google::LogMessage::Flush() at ??:0
@ 0x7f83f6c926d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7f840c830ba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7f840c82f901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f840c83666c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7f840c8364e9 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f83f98bceaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f83fcb966db start_thread at ??:0
@ 0x7f83f5b2c61f clone at ??:0
I20260807 19:42:17.946069 8083 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" instance_seqno: 1786131727626128) as {username='slave'} at 127.6.109.2:34621; Asking this server to re-register.
I20260807 19:42:17.947005 7398 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:17.948868 8083 ts_manager.cc:194] Registered new tserver with Master: 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483)
W20260807 19:42:18.050340 7287 connection.cc:535] client connection to 127.6.109.60:43129 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 19:42:18.050693 7559 connection.cc:535] client connection to 127.6.109.60:43129 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 19:42:18.051486 7559 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.6.109.60:43129)
W20260807 19:42:18.051777 7559 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.6.109.60:43129)
W20260807 19:42:18.052490 7559 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.6.109.60:43129)
W20260807 19:42:18.053313 7559 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.6.109.60:43129)
I20260807 19:42:18.410775 8171 raft_consensus.cc:482] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:18.411064 8171 raft_consensus.cc:504] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:18.413153 8171 leader_election.cc:288] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263), 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129)
W20260807 19:42:18.414691 8070 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:42:18.416425 8070 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:42:18.416738 8070 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129): Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: connect: Connection refused (error 111)
I20260807 19:42:18.416862 8070 leader_election.cc:302] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 959ac7807a8b42a59c61a23d4605c054; no voters: 1b782e30a8d44e7b94f080aa6327134f, 7b31edda0c4a48489ad60fc59c44ed31
I20260807 19:42:18.417273 8171 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 19:42:19.002656 8176 raft_consensus.cc:482] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:19.002900 8176 raft_consensus.cc:504] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:19.003650 8176 leader_election.cc:288] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263), 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129)
W20260807 19:42:19.006685 8070 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:42:19.006943 8070 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129): Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: connect: Connection refused (error 111)
I20260807 19:42:19.007114 8070 leader_election.cc:302] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 959ac7807a8b42a59c61a23d4605c054; no voters: 1b782e30a8d44e7b94f080aa6327134f, 7b31edda0c4a48489ad60fc59c44ed31
I20260807 19:42:19.007375 8176 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 19:42:19.745922 8182 raft_consensus.cc:482] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:19.746124 8182 raft_consensus.cc:504] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:19.746892 8182 leader_election.cc:288] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263), 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129)
W20260807 19:42:19.749181 8070 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:42:19.749440 8070 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129): Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: connect: Connection refused (error 111)
I20260807 19:42:19.749560 8070 leader_election.cc:302] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 959ac7807a8b42a59c61a23d4605c054; no voters: 1b782e30a8d44e7b94f080aa6327134f, 7b31edda0c4a48489ad60fc59c44ed31
I20260807 19:42:19.749827 8182 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 19:42:20.550464 6580 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskeMwJpl/build/asan/bin/kudu
/tmp/dist-test-taskeMwJpl/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.109.60:43129
--webserver_interface=127.6.109.60
--webserver_port=36383
--builtin_ntp_servers=127.6.109.20:40195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129
--hive_metastore_uris=thrift://127.0.0.1:38987
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260807 19:42:20.820570 8189 raft_consensus.cc:482] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:20.820781 8189 raft_consensus.cc:504] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:20.821534 8189 leader_election.cc:288] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263), 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129)
W20260807 19:42:20.833927 8070 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:42:20.834172 8070 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129): Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: connect: Connection refused (error 111)
I20260807 19:42:20.834314 8070 leader_election.cc:302] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 959ac7807a8b42a59c61a23d4605c054; no voters: 1b782e30a8d44e7b94f080aa6327134f, 7b31edda0c4a48489ad60fc59c44ed31
I20260807 19:42:20.834591 8189 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260807 19:42:20.917518 8188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:20.917958 8188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:20.918104 8188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:20.925547 8188 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:20.925740 8188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:20.925832 8188 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:20.925932 8188 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:20.936105 8188 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.109.20:40195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:38987
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.109.60:43129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.109.60
--webserver_port=36383
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:20.937899 8188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:20.940369 8188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:20.955952 8197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:20.956066 8196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:20.956049 8199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:20.957233 8188 server_base.cc:1034] running on GCE node
I20260807 19:42:20.958350 8188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:20.960033 8188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:20.961299 8188 hybrid_clock.cc:648] HybridClock initialized: now 1786131740961212 us; error 63 us; skew 500 ppm
I20260807 19:42:20.961767 8188 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:42:20.965052 8188 webserver.cc:460] Webserver started at http://127.6.109.60:36383/ using document root <none> and password file <none>
I20260807 19:42:20.966008 8188 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:20.966190 8188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:20.973534 8188 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260807 19:42:20.977957 8207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:20.979575 8188 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 19:42:20.979799 8188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data,/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/wal
uuid: "7b31edda0c4a48489ad60fc59c44ed31"
format_stamp: "Formatted at 2026-08-07 19:42:05 on dist-test-slave-p3t5"
I20260807 19:42:20.980892 8188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:21.002348 8188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:21.003394 8188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:21.003746 8188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:21.023865 8188 rpc_server.cc:307] RPC server started. Bound to: 127.6.109.60:43129
I20260807 19:42:21.023872 8261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.109.60:43129 every 8 connection(s)
I20260807 19:42:21.025565 8188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data/info.pb
I20260807 19:42:21.029768 6580 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskeMwJpl/build/asan/bin/kudu as pid 8188
I20260807 19:42:21.033039 8262 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:42:21.037319 8262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: Bootstrap starting.
I20260807 19:42:21.051573 8262 log.cc:824] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:21.068346 7397 heartbeater.cc:347] Connected to a master server at 127.6.109.60:43129
I20260807 19:42:21.076116 7546 heartbeater.cc:347] Connected to a master server at 127.6.109.60:43129
I20260807 19:42:21.084147 8262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: Bootstrap replayed 1/1 log segments. Stats: ops{read=31 overwritten=1 applied=29 ignored=0} inserts{seen=19 ignored=0} mutations{seen=15 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260807 19:42:21.084919 8262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: Bootstrap complete.
I20260807 19:42:21.093122 8262 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 8 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:21.094482 8262 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 8 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b31edda0c4a48489ad60fc59c44ed31, State: Initialized, Role: FOLLOWER
I20260807 19:42:21.095166 8262 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 29, Last appended: 8.30, Last appended by leader: 30, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:21.096101 8268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 8 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:21.096426 8268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:21.097254 8262 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:21.111215 8283 catalog_manager.cc:1259] Loaded cluster ID: a80f7b000fe149f3a73e545a28c78fda
I20260807 19:42:21.111469 8283 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: loading cluster ID for follower catalog manager: success
I20260807 19:42:21.119217 8283 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: acquiring CA information for follower catalog manager: success
I20260807 19:42:21.122658 8283 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 19:42:21.132524 8188 master_runner.cc:186] Error getting master registration for 127.6.109.61:39263: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111),
I20260807 19:42:21.139297 8188 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:42:21.263451 7255 heartbeater.cc:347] Connected to a master server at 127.6.109.60:43129
I20260807 19:42:21.266302 8225 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "051d464d644a4d4486462dfb18274c6a" instance_seqno: 1786131726838357) as {username='slave'} at 127.6.109.1:46879; Asking this server to re-register.
I20260807 19:42:21.267081 7255 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:21.269119 8225 ts_manager.cc:194] Registered new tserver with Master: 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467)
I20260807 19:42:21.573693 8225 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" instance_seqno: 1786131727626128) as {username='slave'} at 127.6.109.2:58659; Asking this server to re-register.
I20260807 19:42:21.574395 7397 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:21.575737 8225 ts_manager.cc:194] Registered new tserver with Master: 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483)
I20260807 19:42:21.579581 8225 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "bafdff0fd60146539631a476c4257e16" instance_seqno: 1786131728431251) as {username='slave'} at 127.6.109.3:54241; Asking this server to re-register.
I20260807 19:42:21.580324 7546 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:21.581724 8225 ts_manager.cc:194] Registered new tserver with Master: bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
I20260807 19:42:21.710750 8289 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:21.711077 8289 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:21.713001 8289 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
I20260807 19:42:21.713707 8093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7b31edda0c4a48489ad60fc59c44ed31" candidate_term: 9 candidate_status { last_received { term: 8 index: 30 } } ignore_live_leader: false dest_uuid: "959ac7807a8b42a59c61a23d4605c054" is_pre_election: true
I20260807 19:42:21.714001 8093 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b31edda0c4a48489ad60fc59c44ed31 in term 8.
W20260807 19:42:21.714376 8211 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:42:21.714668 8210 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31, 959ac7807a8b42a59c61a23d4605c054; no voters:
I20260807 19:42:21.715073 8289 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 19:42:21.715247 8289 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:42:21.715382 8289 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 8 FOLLOWER]: Advancing to term 9
W20260807 19:42:21.716082 8211 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:42:21.720139 8289 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:21.720918 8289 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 9 election: Requested vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
I20260807 19:42:21.721439 8093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7b31edda0c4a48489ad60fc59c44ed31" candidate_term: 9 candidate_status { last_received { term: 8 index: 30 } } ignore_live_leader: false dest_uuid: "959ac7807a8b42a59c61a23d4605c054"
I20260807 19:42:21.721757 8093 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 8 FOLLOWER]: Advancing to term 9
W20260807 19:42:21.722522 8211 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:42:21.723856 8093 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b31edda0c4a48489ad60fc59c44ed31 in term 9.
I20260807 19:42:21.724306 8210 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31, 959ac7807a8b42a59c61a23d4605c054; no voters: 1b782e30a8d44e7b94f080aa6327134f
I20260807 19:42:21.724598 8289 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 9 FOLLOWER]: Leader election won for term 9
I20260807 19:42:21.725046 8289 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 9 LEADER]: Becoming Leader. State: Replica: 7b31edda0c4a48489ad60fc59c44ed31, State: Running, Role: LEADER
I20260807 19:42:21.725543 8289 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 8.30, Last appended by leader: 30, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:21.727767 8292 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b31edda0c4a48489ad60fc59c44ed31. Latest consensus state: current_term: 9 leader_uuid: "7b31edda0c4a48489ad60fc59c44ed31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:21.728057 8292 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:21.728704 8294 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:21.731199 8294 catalog_manager.cc:669] Loaded metadata for table default.table_472c6a33c87c4f1b9cdb926a2e5a6344 [id=0b2f82ef918b47d59b34a5b5757b9486]
I20260807 19:42:21.731663 8294 catalog_manager.cc:669] Loaded metadata for table default.table_c4966a160ae2456ebe5e58c41a9ac129 [id=490e9d5bbd5347aaae98c3a81cb3fe1e]
I20260807 19:42:21.731999 8294 catalog_manager.cc:669] Loaded metadata for table default.table_5d5e7f4976094d7a942e0ba75696c8f7 [id=9918b1d3805d424b92055cf267b8a4ab]
I20260807 19:42:21.732321 8294 catalog_manager.cc:669] Loaded metadata for table default.table_271147f124bf45a88d332b72ac8d3e13 [id=a75161def1a34954bf0846ab30a8cc38]
I20260807 19:42:21.732631 8294 catalog_manager.cc:669] Loaded metadata for table default.table_ca953b708a5a408898f72c483fc31ecf [id=bff436a0a60542b9b0fecacf4a6ed1b6]
I20260807 19:42:21.732962 8294 catalog_manager.cc:669] Loaded metadata for table default.table_49a9f7854b4c4847bce27e16296322b5 [id=cf4da31cb40447199dc079b8b6e709bc]
I20260807 19:42:21.734987 8294 tablet_loader.cc:96] loaded metadata for tablet 248c1670db7244cfac99dd4817b513a4 (table default.table_472c6a33c87c4f1b9cdb926a2e5a6344 [id=0b2f82ef918b47d59b34a5b5757b9486])
I20260807 19:42:21.735281 8294 tablet_loader.cc:96] loaded metadata for tablet 541fb835769b4d1f92407b288847cdee (table default.table_c4966a160ae2456ebe5e58c41a9ac129 [id=490e9d5bbd5347aaae98c3a81cb3fe1e])
I20260807 19:42:21.735565 8294 tablet_loader.cc:96] loaded metadata for tablet 769aecb6d5ca42e09edba0277a022901 (table default.table_5d5e7f4976094d7a942e0ba75696c8f7 [id=9918b1d3805d424b92055cf267b8a4ab])
I20260807 19:42:21.736042 8294 tablet_loader.cc:96] loaded metadata for tablet 807180c74470485182a926ab39d79ab4 (table default.table_ca953b708a5a408898f72c483fc31ecf [id=bff436a0a60542b9b0fecacf4a6ed1b6])
I20260807 19:42:21.736382 8294 tablet_loader.cc:96] loaded metadata for tablet 827e9a16eeb24a2cadf42fbb0b12a872 (table default.table_271147f124bf45a88d332b72ac8d3e13 [id=a75161def1a34954bf0846ab30a8cc38])
I20260807 19:42:21.736704 8294 tablet_loader.cc:96] loaded metadata for tablet 9a45d0765a2c4d93ac9448496ce9c325 (table default.table_49a9f7854b4c4847bce27e16296322b5 [id=cf4da31cb40447199dc079b8b6e709bc])
I20260807 19:42:21.737272 8294 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:21.738447 8294 catalog_manager.cc:1259] Loaded cluster ID: a80f7b000fe149f3a73e545a28c78fda
I20260807 19:42:21.738550 8294 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:21.739813 8294 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:21.740727 8294 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: Loaded TSK: 0
I20260807 19:42:21.741698 8294 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:42:21.742578 8294 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260807 19:42:21.743446 8294 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260807 19:42:21.811681 8225 master_service.cc:946] ReplaceTablet: received request to replace tablet d328d2e772ed4a6bb8597658f907f0f0 from {username='slave'} at 127.0.0.1:42658
I20260807 19:42:21.812233 8223 master_service.cc:946] ReplaceTablet: received request to replace tablet 9a45d0765a2c4d93ac9448496ce9c325 from {username='slave'} at 127.0.0.1:42658
I20260807 19:42:21.812575 8224 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42658:
name: "default.Table_c471425540714f6e9c8e61a7af3470cc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:21.813081 8218 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:42658:
table { table_name: "default.Table_472c6a33c87c4f1b9cdb926a2e5a6344" } new_table_name: "default.Table_cda45f853cca4f52a3371bbaec9c7750" modify_external_catalogs: true
I20260807 19:42:21.813367 8221 master_service.cc:946] ReplaceTablet: received request to replace tablet d328d2e772ed4a6bb8597658f907f0f0 from {username='slave'} at 127.0.0.1:42658
I20260807 19:42:21.813690 8220 master_service.cc:946] ReplaceTablet: received request to replace tablet 0d47a7a1285948468c0e6e1bddcd26d4 from {username='slave'} at 127.0.0.1:42658
I20260807 19:42:21.814483 8221 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42658:
name: "default.Table_1455c4f504ff4603bc11da85f16cc69f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:21.815290 8220 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42658:
name: "default.Table_61514c4f10b54148a57bbaf295205ace"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:21.815825 8225 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42658:
name: "default.Table_5d300b1b9ec343b88bbb7481707c963a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:21.816718 8222 master_service.cc:946] ReplaceTablet: received request to replace tablet 248c1670db7244cfac99dd4817b513a4 from {username='slave'} at 127.0.0.1:42658
I20260807 19:42:21.817693 8219 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42658:
name: "default.Table_f2d1ca595d0c43ec982c1377f39155ee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:21.818171 8217 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42658:
name: "default.Table_8cc3d0e85b19426f8df95f1894914926"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:21.817476 8215 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42658:
name: "default.Table_b6eca8c0167f4d80b6ef4f8125aea68f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260807 19:42:21.822057 8282 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_c4966a160ae2456ebe5e58c41a9ac129
*** Check failure stack trace: ***
@ 0x7f927ab779ad google::LogMessage::Fail() at ??:0
I20260807 19:42:21.824898 8093 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 9 FOLLOWER]: Refusing update from remote peer 7b31edda0c4a48489ad60fc59c44ed31: Log matching property violated. Preceding OpId in replica: term: 6 index: 30. Preceding OpId from leader: term: 9 index: 32. (index mismatch)
I20260807 19:42:21.826136 8292 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 25, Time since last communication: 0.000s
I20260807 19:42:21.826531 8292 consensus_queue.cc:1225] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [LEADER]: Peer 959ac7807a8b42a59c61a23d4605c054 log is divergent from this leader: its last log entry 6.30 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 25
@ 0x7f927ab7a88a google::LogMessage::SendToLog() at ??:0
@ 0x7f927ab774de google::LogMessage::Flush() at ??:0
I20260807 19:42:21.829481 8093 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 9 FOLLOWER]: Deduplicated request from leader. Original: 5.25->[6.26-9.33] Dedup: 6.28->[7.29-9.33]
@ 0x7f927ab776d1 google::LogMessage::~LogMessage() at ??:0
I20260807 19:42:21.829830 8093 pending_rounds.cc:76] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054: Aborting all ops after (but not including) 28
I20260807 19:42:21.829958 8093 pending_rounds.cc:97] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054: Aborting uncommitted WRITE_OP operation due to leader change: 6.29
I20260807 19:42:21.830641 8093 pending_rounds.cc:97] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054: Aborting uncommitted WRITE_OP operation due to leader change: 6.30
W20260807 19:42:21.838231 8211 consensus_peers.cc:487] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 -> Peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Couldn't send request to peer 1b782e30a8d44e7b94f080aa6327134f. Status: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
@ 0x7f9290715ba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7f9290714901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f929071b66c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7f929071b4e9 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f927d7a1eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f9280a7b6db start_thread at ??:0
@ 0x7f9279a1161f clone at ??:0
I20260807 19:42:21.894264 8313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 9 leader_uuid: "7b31edda0c4a48489ad60fc59c44ed31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:21.894666 8313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:21.895124 8304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 9 leader_uuid: "7b31edda0c4a48489ad60fc59c44ed31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:21.895507 8304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:21.895853 8298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b31edda0c4a48489ad60fc59c44ed31. Latest consensus state: current_term: 9 leader_uuid: "7b31edda0c4a48489ad60fc59c44ed31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:21.896195 8298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 19:42:22.072067 7559 connection.cc:535] client connection to 127.6.109.60:43129 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 19:42:22.072023 8067 connection.cc:535] server connection from 127.0.0.1:45124 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 19:42:22.072834 7559 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.6.109.60:43129)
W20260807 19:42:22.073385 7559 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.6.109.60:43129)
W20260807 19:42:22.073832 7559 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.6.109.60:43129)
W20260807 19:42:22.074083 7559 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.6.109.60:43129)
I20260807 19:42:22.273288 7255 heartbeater.cc:503] Master 127.6.109.60:43129 was elected leader, sending a full tablet report...
I20260807 19:42:22.336660 8298 raft_consensus.cc:482] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7b31edda0c4a48489ad60fc59c44ed31)
I20260807 19:42:22.336999 8298 raft_consensus.cc:504] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:22.337786 8298 leader_election.cc:288] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263), 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129)
W20260807 19:42:22.341150 8070 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:42:22.341373 8070 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129): Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: connect: Connection refused (error 111)
I20260807 19:42:22.341485 8070 leader_election.cc:302] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 959ac7807a8b42a59c61a23d4605c054; no voters: 1b782e30a8d44e7b94f080aa6327134f, 7b31edda0c4a48489ad60fc59c44ed31
I20260807 19:42:22.341786 8298 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260807 19:42:22.503464 8044 master_runner.cc:186] Error getting master registration for 127.6.109.61:39263: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111),
I20260807 19:42:22.503671 8044 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:42:22.924506 8327 raft_consensus.cc:482] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7b31edda0c4a48489ad60fc59c44ed31)
I20260807 19:42:22.924695 8327 raft_consensus.cc:504] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:22.925316 8327 leader_election.cc:288] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263), 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129)
W20260807 19:42:22.927606 8070 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:42:22.927858 8070 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129): Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: connect: Connection refused (error 111)
I20260807 19:42:22.928025 8070 leader_election.cc:302] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 959ac7807a8b42a59c61a23d4605c054; no voters: 1b782e30a8d44e7b94f080aa6327134f, 7b31edda0c4a48489ad60fc59c44ed31
I20260807 19:42:22.928283 8327 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260807 19:42:23.419940 6580 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskeMwJpl/build/asan/bin/kudu pid 8044
I20260807 19:42:23.433986 8113 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:23.553580 6580 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskeMwJpl/build/asan/bin/kudu with pid 8044
I20260807 19:42:24.156176 6580 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskeMwJpl/build/asan/bin/kudu
/tmp/dist-test-taskeMwJpl/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.109.62:43417
--webserver_interface=127.6.109.62
--webserver_port=37311
--builtin_ntp_servers=127.6.109.20:40195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129
--hive_metastore_uris=thrift://127.0.0.1:38987
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260807 19:42:24.516010 8336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:24.516340 8336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:24.516422 8336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:24.523919 8336 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:24.524052 8336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:24.524096 8336 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:24.524158 8336 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:24.534312 8336 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.109.20:40195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:38987
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.109.62:43417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.109.62
--webserver_port=37311
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:24.535704 8336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:24.537760 8336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:24.551833 8341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:24.551810 8342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:24.551793 8344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:24.552807 8336 server_base.cc:1034] running on GCE node
I20260807 19:42:24.553848 8336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:24.555392 8336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:24.556677 8336 hybrid_clock.cc:648] HybridClock initialized: now 1786131744556554 us; error 77 us; skew 500 ppm
I20260807 19:42:24.557175 8336 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:42:24.568923 8336 webserver.cc:460] Webserver started at http://127.6.109.62:37311/ using document root <none> and password file <none>
I20260807 19:42:24.570092 8336 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:24.570271 8336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:24.577006 8336 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 19:42:24.580862 8350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:24.582458 8336 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260807 19:42:24.582690 8336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/data,/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/wal
uuid: "959ac7807a8b42a59c61a23d4605c054"
format_stamp: "Formatted at 2026-08-07 19:42:04 on dist-test-slave-p3t5"
I20260807 19:42:24.583808 8336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:24.603104 8336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:24.604206 8336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:24.604521 8336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:24.623164 8336 rpc_server.cc:307] RPC server started. Bound to: 127.6.109.62:43417
I20260807 19:42:24.623176 8403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.109.62:43417 every 8 connection(s)
I20260807 19:42:24.624737 8336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/data/info.pb
I20260807 19:42:24.630188 6580 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskeMwJpl/build/asan/bin/kudu as pid 8336
I20260807 19:42:24.632354 8404 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:42:24.636832 8404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054: Bootstrap starting.
I20260807 19:42:24.650309 8404 log.cc:824] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:24.675007 8404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054: Bootstrap replayed 1/1 log segments. Stats: ops{read=35 overwritten=2 applied=29 ignored=0} inserts{seen=19 ignored=0} mutations{seen=15 ignored=0} orphaned_commits=0. Pending: 4 replicates
I20260807 19:42:24.675714 8404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054: Bootstrap complete.
I20260807 19:42:24.684325 8404 raft_consensus.cc:348] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 9 FOLLOWER]: Replica starting. Triggering 4 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:24.689248 8404 raft_consensus.cc:729] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 9 FOLLOWER]: Becoming Follower/Learner. State: Replica: 959ac7807a8b42a59c61a23d4605c054, State: Initialized, Role: FOLLOWER
I20260807 19:42:24.690346 8404 consensus_queue.cc:260] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 29, Last appended: 9.33, Last appended by leader: 33, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:24.691291 8409 sys_catalog.cc:455] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 9 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:24.691645 8409 sys_catalog.cc:458] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:24.692557 8404 sys_catalog.cc:564] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:24.712242 8423 catalog_manager.cc:1259] Loaded cluster ID: a80f7b000fe149f3a73e545a28c78fda
I20260807 19:42:24.712352 8423 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054: loading cluster ID for follower catalog manager: success
I20260807 19:42:24.718958 8423 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054: acquiring CA information for follower catalog manager: success
I20260807 19:42:24.721881 8423 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 19:42:24.723333 8336 master_runner.cc:186] Error getting master registration for 127.6.109.61:39263: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111),
I20260807 19:42:24.729455 8336 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:42:24.775715 7256 heartbeater.cc:347] Connected to a master server at 127.6.109.62:43417
I20260807 19:42:24.778977 8367 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "051d464d644a4d4486462dfb18274c6a" instance_seqno: 1786131726838357) as {username='slave'} at 127.6.109.1:38687; Asking this server to re-register.
I20260807 19:42:24.779812 7256 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:24.782157 8367 ts_manager.cc:194] Registered new tserver with Master: 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467)
I20260807 19:42:24.914912 7547 heartbeater.cc:347] Connected to a master server at 127.6.109.62:43417
I20260807 19:42:24.916316 8367 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "bafdff0fd60146539631a476c4257e16" instance_seqno: 1786131728431251) as {username='slave'} at 127.6.109.3:33761; Asking this server to re-register.
I20260807 19:42:24.916949 7547 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:24.918318 8367 ts_manager.cc:194] Registered new tserver with Master: bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
I20260807 19:42:25.088589 7398 heartbeater.cc:347] Connected to a master server at 127.6.109.62:43417
I20260807 19:42:25.090593 8367 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" instance_seqno: 1786131727626128) as {username='slave'} at 127.6.109.2:43869; Asking this server to re-register.
I20260807 19:42:25.091233 7398 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:25.092656 8367 ts_manager.cc:194] Registered new tserver with Master: 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483)
I20260807 19:42:25.315090 8431 raft_consensus.cc:482] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:25.315439 8431 raft_consensus.cc:504] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:25.317445 8431 leader_election.cc:288] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263), 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129)
W20260807 19:42:25.318923 8354 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:42:25.320490 8354 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:42:25.320777 8354 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129): Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: connect: Connection refused (error 111)
I20260807 19:42:25.320896 8354 leader_election.cc:302] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 959ac7807a8b42a59c61a23d4605c054; no voters: 1b782e30a8d44e7b94f080aa6327134f, 7b31edda0c4a48489ad60fc59c44ed31
I20260807 19:42:25.321305 8431 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260807 19:42:26.009670 8437 raft_consensus.cc:482] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:26.010022 8437 raft_consensus.cc:504] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:26.010759 8437 leader_election.cc:288] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263), 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129)
W20260807 19:42:26.014869 8354 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:42:26.015128 8354 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129): Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: connect: Connection refused (error 111)
I20260807 19:42:26.015269 8354 leader_election.cc:302] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 959ac7807a8b42a59c61a23d4605c054; no voters: 1b782e30a8d44e7b94f080aa6327134f, 7b31edda0c4a48489ad60fc59c44ed31
I20260807 19:42:26.015599 8437 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260807 19:42:27.116957 8443 raft_consensus.cc:482] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:27.117189 8443 raft_consensus.cc:504] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:27.117983 8443 leader_election.cc:288] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263), 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129)
W20260807 19:42:27.120091 8354 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:42:27.120292 8354 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129): Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: connect: Connection refused (error 111)
I20260807 19:42:27.120373 8354 leader_election.cc:302] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 959ac7807a8b42a59c61a23d4605c054; no voters: 1b782e30a8d44e7b94f080aa6327134f, 7b31edda0c4a48489ad60fc59c44ed31
I20260807 19:42:27.120623 8443 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260807 19:42:27.877422 8448 raft_consensus.cc:482] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:27.877676 8448 raft_consensus.cc:504] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:27.878271 8448 leader_election.cc:288] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263), 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129)
W20260807 19:42:27.881543 8354 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:42:27.881840 8354 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129): Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: connect: Connection refused (error 111)
I20260807 19:42:27.881975 8354 leader_election.cc:302] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 959ac7807a8b42a59c61a23d4605c054; no voters: 1b782e30a8d44e7b94f080aa6327134f, 7b31edda0c4a48489ad60fc59c44ed31
I20260807 19:42:27.882287 8448 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260807 19:42:28.050213 6580 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskeMwJpl/build/asan/bin/kudu
/tmp/dist-test-taskeMwJpl/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.109.60:43129
--webserver_interface=127.6.109.60
--webserver_port=36383
--builtin_ntp_servers=127.6.109.20:40195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129
--hive_metastore_uris=thrift://127.0.0.1:38987
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260807 19:42:28.414378 8454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:28.414790 8454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:28.414951 8454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:28.422878 8454 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:28.423053 8454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:28.423166 8454 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:28.423271 8454 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:28.433514 8454 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.109.20:40195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:38987
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.109.60:43129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.109.60
--webserver_port=36383
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:28.435333 8454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:28.437512 8454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:28.450909 8460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:28.451053 8462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:28.451304 8459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:28.452517 8454 server_base.cc:1034] running on GCE node
I20260807 19:42:28.453809 8454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:28.455886 8454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:28.457175 8454 hybrid_clock.cc:648] HybridClock initialized: now 1786131748457072 us; error 69 us; skew 500 ppm
I20260807 19:42:28.457649 8454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:42:28.468611 8454 webserver.cc:460] Webserver started at http://127.6.109.60:36383/ using document root <none> and password file <none>
I20260807 19:42:28.469507 8454 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:28.469720 8454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:28.475725 8454 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 19:42:28.479022 8468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:28.480499 8454 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:42:28.480718 8454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data,/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/wal
uuid: "7b31edda0c4a48489ad60fc59c44ed31"
format_stamp: "Formatted at 2026-08-07 19:42:05 on dist-test-slave-p3t5"
I20260807 19:42:28.481904 8454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:28.498672 8454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:28.499702 8454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:28.500051 8454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:28.520190 8454 rpc_server.cc:307] RPC server started. Bound to: 127.6.109.60:43129
I20260807 19:42:28.520208 8520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.109.60:43129 every 8 connection(s)
I20260807 19:42:28.521899 8454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data/info.pb
I20260807 19:42:28.524819 6580 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskeMwJpl/build/asan/bin/kudu as pid 8454
I20260807 19:42:28.530898 8521 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:42:28.535967 8521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: Bootstrap starting.
I20260807 19:42:28.550582 8521 log.cc:824] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:28.580073 8521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: Bootstrap replayed 1/1 log segments. Stats: ops{read=34 overwritten=1 applied=29 ignored=0} inserts{seen=19 ignored=0} mutations{seen=15 ignored=0} orphaned_commits=0. Pending: 4 replicates
I20260807 19:42:28.580794 8521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: Bootstrap complete.
I20260807 19:42:28.588894 8521 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 9 FOLLOWER]: Replica starting. Triggering 4 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:28.592921 8521 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 9 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b31edda0c4a48489ad60fc59c44ed31, State: Initialized, Role: FOLLOWER
I20260807 19:42:28.593768 8521 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 29, Last appended: 9.33, Last appended by leader: 33, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:28.594916 8526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 9 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:28.595490 8526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:28.596436 8521 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:28.604230 7397 heartbeater.cc:347] Connected to a master server at 127.6.109.60:43129
I20260807 19:42:28.611287 7546 heartbeater.cc:347] Connected to a master server at 127.6.109.60:43129
I20260807 19:42:28.616917 8485 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "bafdff0fd60146539631a476c4257e16" instance_seqno: 1786131728431251) as {username='slave'} at 127.6.109.3:35237; Asking this server to re-register.
I20260807 19:42:28.618180 7546 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:28.620755 8485 ts_manager.cc:194] Registered new tserver with Master: bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
I20260807 19:42:28.654954 8542 catalog_manager.cc:1259] Loaded cluster ID: a80f7b000fe149f3a73e545a28c78fda
I20260807 19:42:28.655160 8542 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: loading cluster ID for follower catalog manager: success
I20260807 19:42:28.659906 8542 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: acquiring CA information for follower catalog manager: success
I20260807 19:42:28.662968 8542 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 19:42:28.653105 8454 master_runner.cc:186] Error getting master registration for 127.6.109.61:39263: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111),
I20260807 19:42:28.666523 8454 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:42:28.800102 7255 heartbeater.cc:347] Connected to a master server at 127.6.109.60:43129
I20260807 19:42:28.800362 7255 heartbeater.cc:503] Master 127.6.109.60:43129 was elected leader, sending a full tablet report...
I20260807 19:42:28.803107 8485 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "051d464d644a4d4486462dfb18274c6a" instance_seqno: 1786131726838357) as {username='slave'} at 127.6.109.1:60853; Asking this server to re-register.
I20260807 19:42:28.803793 7255 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:28.805392 8485 ts_manager.cc:194] Registered new tserver with Master: 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467)
I20260807 19:42:29.059046 8526 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:29.059445 8526 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:29.061544 8526 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:42:29.062352 8472 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:42:29.062278 8377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7b31edda0c4a48489ad60fc59c44ed31" candidate_term: 10 candidate_status { last_received { term: 9 index: 33 } } ignore_live_leader: false dest_uuid: "959ac7807a8b42a59c61a23d4605c054" is_pre_election: true
I20260807 19:42:29.062808 8377 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b31edda0c4a48489ad60fc59c44ed31 in term 9.
I20260807 19:42:29.063571 8471 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31, 959ac7807a8b42a59c61a23d4605c054; no voters:
I20260807 19:42:29.063982 8526 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260807 19:42:29.064203 8526 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:42:29.064395 8526 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 9 FOLLOWER]: Advancing to term 10
W20260807 19:42:29.064375 8472 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:42:29.068717 8526 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:29.069454 8526 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 10 election: Requested vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
I20260807 19:42:29.070011 8377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7b31edda0c4a48489ad60fc59c44ed31" candidate_term: 10 candidate_status { last_received { term: 9 index: 33 } } ignore_live_leader: false dest_uuid: "959ac7807a8b42a59c61a23d4605c054"
I20260807 19:42:29.070252 8377 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 9 FOLLOWER]: Advancing to term 10
W20260807 19:42:29.071065 8472 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:42:29.074141 8377 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b31edda0c4a48489ad60fc59c44ed31 in term 10.
I20260807 19:42:29.074626 8471 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31, 959ac7807a8b42a59c61a23d4605c054; no voters: 1b782e30a8d44e7b94f080aa6327134f
I20260807 19:42:29.075013 8526 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 10 FOLLOWER]: Leader election won for term 10
I20260807 19:42:29.075479 8526 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 10 LEADER]: Becoming Leader. State: Replica: 7b31edda0c4a48489ad60fc59c44ed31, State: Running, Role: LEADER
I20260807 19:42:29.075973 8526 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 9.33, Last appended by leader: 33, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:29.078758 8550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b31edda0c4a48489ad60fc59c44ed31. Latest consensus state: current_term: 10 leader_uuid: "7b31edda0c4a48489ad60fc59c44ed31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:29.079188 8550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:29.109146 8485 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" instance_seqno: 1786131727626128) as {username='slave'} at 127.6.109.2:35643; Asking this server to re-register.
I20260807 19:42:29.109958 7397 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:29.111339 8485 ts_manager.cc:194] Registered new tserver with Master: 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483)
I20260807 19:42:29.557482 8377 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 10 FOLLOWER]: Refusing update from remote peer 7b31edda0c4a48489ad60fc59c44ed31: Log matching property violated. Preceding OpId in replica: term: 9 index: 33. Preceding OpId from leader: term: 10 index: 34. (index mismatch)
I20260807 19:42:29.558636 8550 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 29, Time since last communication: 0.000s
I20260807 19:42:29.562805 8558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b31edda0c4a48489ad60fc59c44ed31. Latest consensus state: current_term: 10 leader_uuid: "7b31edda0c4a48489ad60fc59c44ed31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:29.563198 8558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:29.568706 8550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 10 leader_uuid: "7b31edda0c4a48489ad60fc59c44ed31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:29.569087 8550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:29.569339 8550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 10 leader_uuid: "7b31edda0c4a48489ad60fc59c44ed31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:29.569675 8550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:29.570089 8562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 10 leader_uuid: "7b31edda0c4a48489ad60fc59c44ed31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:29.570510 8562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: This master's current role is: LEADER
W20260807 19:42:29.570686 8472 consensus_peers.cc:487] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 -> Peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Couldn't send request to peer 1b782e30a8d44e7b94f080aa6327134f. Status: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 19:42:29.572290 8558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 10 leader_uuid: "7b31edda0c4a48489ad60fc59c44ed31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:29.572265 8564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 10 leader_uuid: "7b31edda0c4a48489ad60fc59c44ed31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:29.572693 8558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:29.572721 8564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:29.573025 8570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 10 leader_uuid: "7b31edda0c4a48489ad60fc59c44ed31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:29.573305 8570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:29.596453 8553 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:29.599982 8553 catalog_manager.cc:669] Loaded metadata for table default.table_472c6a33c87c4f1b9cdb926a2e5a6344 [id=0b2f82ef918b47d59b34a5b5757b9486]
I20260807 19:42:29.600572 8553 catalog_manager.cc:669] Loaded metadata for table default.table_c4966a160ae2456ebe5e58c41a9ac129 [id=490e9d5bbd5347aaae98c3a81cb3fe1e]
I20260807 19:42:29.600950 8553 catalog_manager.cc:669] Loaded metadata for table default.table_5d5e7f4976094d7a942e0ba75696c8f7 [id=9918b1d3805d424b92055cf267b8a4ab]
I20260807 19:42:29.601310 8553 catalog_manager.cc:669] Loaded metadata for table default.table_271147f124bf45a88d332b72ac8d3e13 [id=a75161def1a34954bf0846ab30a8cc38]
I20260807 19:42:29.601759 8553 catalog_manager.cc:669] Loaded metadata for table default.table_ca953b708a5a408898f72c483fc31ecf [id=bff436a0a60542b9b0fecacf4a6ed1b6]
I20260807 19:42:29.602212 8553 catalog_manager.cc:669] Loaded metadata for table default.table_49a9f7854b4c4847bce27e16296322b5 [id=cf4da31cb40447199dc079b8b6e709bc]
I20260807 19:42:29.604589 8553 tablet_loader.cc:96] loaded metadata for tablet 541fb835769b4d1f92407b288847cdee (table default.table_c4966a160ae2456ebe5e58c41a9ac129 [id=490e9d5bbd5347aaae98c3a81cb3fe1e])
I20260807 19:42:29.604951 8553 tablet_loader.cc:96] loaded metadata for tablet 769aecb6d5ca42e09edba0277a022901 (table default.table_5d5e7f4976094d7a942e0ba75696c8f7 [id=9918b1d3805d424b92055cf267b8a4ab])
I20260807 19:42:29.605286 8553 tablet_loader.cc:96] loaded metadata for tablet 807180c74470485182a926ab39d79ab4 (table default.table_ca953b708a5a408898f72c483fc31ecf [id=bff436a0a60542b9b0fecacf4a6ed1b6])
I20260807 19:42:29.605563 8553 tablet_loader.cc:96] loaded metadata for tablet 827e9a16eeb24a2cadf42fbb0b12a872 (table default.table_271147f124bf45a88d332b72ac8d3e13 [id=a75161def1a34954bf0846ab30a8cc38])
I20260807 19:42:29.606204 8553 tablet_loader.cc:96] loaded metadata for tablet f485a2f4a2a6455e8f15c73692c37dc3 (table default.table_472c6a33c87c4f1b9cdb926a2e5a6344 [id=0b2f82ef918b47d59b34a5b5757b9486])
I20260807 19:42:29.606472 8553 tablet_loader.cc:96] loaded metadata for tablet fbea08c1f63a4a1c8a4d093be38f870e (table default.table_49a9f7854b4c4847bce27e16296322b5 [id=cf4da31cb40447199dc079b8b6e709bc])
I20260807 19:42:29.606809 8553 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:29.608829 8553 catalog_manager.cc:1259] Loaded cluster ID: a80f7b000fe149f3a73e545a28c78fda
I20260807 19:42:29.608924 8553 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:29.610293 8553 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:29.611231 8553 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: Loaded TSK: 0
I20260807 19:42:29.612447 8553 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:42:29.613751 8553 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260807 19:42:29.614991 8553 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260807 19:42:29.685178 7331 tablet_service.cc:1467] Processing CreateTablet for tablet fbea08c1f63a4a1c8a4d093be38f870e (DEFAULT_TABLE table=default.table_49a9f7854b4c4847bce27e16296322b5 [id=cf4da31cb40447199dc079b8b6e709bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:29.685449 7330 tablet_service.cc:1467] Processing CreateTablet for tablet f485a2f4a2a6455e8f15c73692c37dc3 (DEFAULT_TABLE table=default.table_472c6a33c87c4f1b9cdb926a2e5a6344 [id=0b2f82ef918b47d59b34a5b5757b9486]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:29.685810 7331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbea08c1f63a4a1c8a4d093be38f870e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:29.686115 7330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f485a2f4a2a6455e8f15c73692c37dc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:29.686215 7328 tablet_service.cc:1467] Processing CreateTablet for tablet 541fb835769b4d1f92407b288847cdee (DEFAULT_TABLE table=default.table_c4966a160ae2456ebe5e58c41a9ac129 [id=490e9d5bbd5347aaae98c3a81cb3fe1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:29.686470 7332 tablet_service.cc:1467] Processing CreateTablet for tablet 827e9a16eeb24a2cadf42fbb0b12a872 (DEFAULT_TABLE table=default.table_271147f124bf45a88d332b72ac8d3e13 [id=a75161def1a34954bf0846ab30a8cc38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:29.686708 7328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 541fb835769b4d1f92407b288847cdee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:29.687625 7187 tablet_service.cc:1467] Processing CreateTablet for tablet fbea08c1f63a4a1c8a4d093be38f870e (DEFAULT_TABLE table=default.table_49a9f7854b4c4847bce27e16296322b5 [id=cf4da31cb40447199dc079b8b6e709bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:29.687901 7186 tablet_service.cc:1467] Processing CreateTablet for tablet 827e9a16eeb24a2cadf42fbb0b12a872 (DEFAULT_TABLE table=default.table_271147f124bf45a88d332b72ac8d3e13 [id=a75161def1a34954bf0846ab30a8cc38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:29.688090 7187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbea08c1f63a4a1c8a4d093be38f870e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:29.688340 7186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 827e9a16eeb24a2cadf42fbb0b12a872. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:29.690855 7480 tablet_service.cc:1467] Processing CreateTablet for tablet fbea08c1f63a4a1c8a4d093be38f870e (DEFAULT_TABLE table=default.table_49a9f7854b4c4847bce27e16296322b5 [id=cf4da31cb40447199dc079b8b6e709bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:29.691352 7480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbea08c1f63a4a1c8a4d093be38f870e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:29.692126 7185 tablet_service.cc:1467] Processing CreateTablet for tablet f485a2f4a2a6455e8f15c73692c37dc3 (DEFAULT_TABLE table=default.table_472c6a33c87c4f1b9cdb926a2e5a6344 [id=0b2f82ef918b47d59b34a5b5757b9486]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:29.692523 7185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f485a2f4a2a6455e8f15c73692c37dc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:29.694073 7479 tablet_service.cc:1467] Processing CreateTablet for tablet f485a2f4a2a6455e8f15c73692c37dc3 (DEFAULT_TABLE table=default.table_472c6a33c87c4f1b9cdb926a2e5a6344 [id=0b2f82ef918b47d59b34a5b5757b9486]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:29.694553 7479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f485a2f4a2a6455e8f15c73692c37dc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:29.698035 7481 tablet_service.cc:1467] Processing CreateTablet for tablet 827e9a16eeb24a2cadf42fbb0b12a872 (DEFAULT_TABLE table=default.table_271147f124bf45a88d332b72ac8d3e13 [id=a75161def1a34954bf0846ab30a8cc38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:29.698103 7184 tablet_service.cc:1467] Processing CreateTablet for tablet 541fb835769b4d1f92407b288847cdee (DEFAULT_TABLE table=default.table_c4966a160ae2456ebe5e58c41a9ac129 [id=490e9d5bbd5347aaae98c3a81cb3fe1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:29.698511 7481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 827e9a16eeb24a2cadf42fbb0b12a872. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:29.698529 7184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 541fb835769b4d1f92407b288847cdee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:29.690860 7478 tablet_service.cc:1467] Processing CreateTablet for tablet 541fb835769b4d1f92407b288847cdee (DEFAULT_TABLE table=default.table_c4966a160ae2456ebe5e58c41a9ac129 [id=490e9d5bbd5347aaae98c3a81cb3fe1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:29.702363 7478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 541fb835769b4d1f92407b288847cdee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:29.702807 7332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 827e9a16eeb24a2cadf42fbb0b12a872. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:29.703953 8577 tablet_bootstrap.cc:492] T fbea08c1f63a4a1c8a4d093be38f870e P 051d464d644a4d4486462dfb18274c6a: Bootstrap starting.
I20260807 19:42:29.706802 8577 tablet_bootstrap.cc:654] T fbea08c1f63a4a1c8a4d093be38f870e P 051d464d644a4d4486462dfb18274c6a: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:29.710882 8577 tablet_bootstrap.cc:492] T fbea08c1f63a4a1c8a4d093be38f870e P 051d464d644a4d4486462dfb18274c6a: No bootstrap required, opened a new log
I20260807 19:42:29.711074 8577 ts_tablet_manager.cc:1397] T fbea08c1f63a4a1c8a4d093be38f870e P 051d464d644a4d4486462dfb18274c6a: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260807 19:42:29.711794 8577 raft_consensus.cc:348] T fbea08c1f63a4a1c8a4d093be38f870e P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } }
I20260807 19:42:29.712045 8577 raft_consensus.cc:374] T fbea08c1f63a4a1c8a4d093be38f870e P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:29.712134 8577 raft_consensus.cc:729] T fbea08c1f63a4a1c8a4d093be38f870e P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 051d464d644a4d4486462dfb18274c6a, State: Initialized, Role: FOLLOWER
I20260807 19:42:29.712419 8577 consensus_queue.cc:260] T fbea08c1f63a4a1c8a4d093be38f870e P 051d464d644a4d4486462dfb18274c6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } }
I20260807 19:42:29.713021 8577 ts_tablet_manager.cc:1428] T fbea08c1f63a4a1c8a4d093be38f870e P 051d464d644a4d4486462dfb18274c6a: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260807 19:42:29.722451 7255 heartbeater.cc:503] Master 127.6.109.60:43129 was elected leader, sending a full tablet report...
I20260807 19:42:29.727028 8577 tablet_bootstrap.cc:492] T 827e9a16eeb24a2cadf42fbb0b12a872 P 051d464d644a4d4486462dfb18274c6a: Bootstrap starting.
I20260807 19:42:29.727617 8576 tablet_bootstrap.cc:492] T f485a2f4a2a6455e8f15c73692c37dc3 P 35b5ecc2c9ef47c2bbe35653633e14da: Bootstrap starting.
I20260807 19:42:29.728370 8485 catalog_manager.cc:5495] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a reported cstate change: term changed from 0 to 1, leader changed from <none> to 051d464d644a4d4486462dfb18274c6a (127.6.109.1). New cstate: current_term: 1 leader_uuid: "051d464d644a4d4486462dfb18274c6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } health_report { overall_health: HEALTHY } } }
I20260807 19:42:29.728963 8577 tablet_bootstrap.cc:654] T 827e9a16eeb24a2cadf42fbb0b12a872 P 051d464d644a4d4486462dfb18274c6a: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:29.729506 8485 catalog_manager.cc:5495] T 769aecb6d5ca42e09edba0277a022901 P 051d464d644a4d4486462dfb18274c6a reported cstate change: term changed from 0 to 1, leader changed from <none> to 051d464d644a4d4486462dfb18274c6a (127.6.109.1). New cstate: current_term: 1 leader_uuid: "051d464d644a4d4486462dfb18274c6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } health_report { overall_health: HEALTHY } } }
I20260807 19:42:29.729806 8576 tablet_bootstrap.cc:654] T f485a2f4a2a6455e8f15c73692c37dc3 P 35b5ecc2c9ef47c2bbe35653633e14da: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:29.738922 8576 tablet_bootstrap.cc:492] T f485a2f4a2a6455e8f15c73692c37dc3 P 35b5ecc2c9ef47c2bbe35653633e14da: No bootstrap required, opened a new log
I20260807 19:42:29.739221 8576 ts_tablet_manager.cc:1397] T f485a2f4a2a6455e8f15c73692c37dc3 P 35b5ecc2c9ef47c2bbe35653633e14da: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.008s
I20260807 19:42:29.740329 8482 master_service.cc:946] ReplaceTablet: received request to replace tablet 769aecb6d5ca42e09edba0277a022901 from {username='slave'} at 127.0.0.1:56494
I20260807 19:42:29.740996 8576 raft_consensus.cc:348] T f485a2f4a2a6455e8f15c73692c37dc3 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:42:29.741452 8576 raft_consensus.cc:374] T f485a2f4a2a6455e8f15c73692c37dc3 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:29.741667 8576 raft_consensus.cc:729] T f485a2f4a2a6455e8f15c73692c37dc3 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35b5ecc2c9ef47c2bbe35653633e14da, State: Initialized, Role: FOLLOWER
I20260807 19:42:29.742084 8576 consensus_queue.cc:260] T f485a2f4a2a6455e8f15c73692c37dc3 P 35b5ecc2c9ef47c2bbe35653633e14da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:42:29.742439 8484 master_service.cc:946] ReplaceTablet: received request to replace tablet 248c1670db7244cfac99dd4817b513a4 from {username='slave'} at 127.0.0.1:56494
I20260807 19:42:29.743533 8483 master_service.cc:946] ReplaceTablet: received request to replace tablet 0d47a7a1285948468c0e6e1bddcd26d4 from {username='slave'} at 127.0.0.1:56494
I20260807 19:42:29.745035 8583 tablet_bootstrap.cc:492] T fbea08c1f63a4a1c8a4d093be38f870e P bafdff0fd60146539631a476c4257e16: Bootstrap starting.
I20260807 19:42:29.744805 8483 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56494:
name: "default.Table_c471425540714f6e9c8e61a7af3470cc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:29.746106 8481 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56494:
name: "default.Table_24957fcd29dc4a3db1b0ab2dcf131779"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:29.746402 7184 tablet_service.cc:1514] Processing DeleteTablet for tablet 248c1670db7244cfac99dd4817b513a4 with delete_type TABLET_DATA_DELETED (replaced by tablet f485a2f4a2a6455e8f15c73692c37dc3) from {username='slave'} at 127.0.0.1:44714
I20260807 19:42:29.746882 8480 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56494:
name: "default.Table_8ea588612f6c47c79ac5c31aa592809e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:29.747159 7185 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a45d0765a2c4d93ac9448496ce9c325 with delete_type TABLET_DATA_DELETED (replaced by tablet fbea08c1f63a4a1c8a4d093be38f870e) from {username='slave'} at 127.0.0.1:44714
I20260807 19:42:29.747582 7185 tablet_service.cc:1514] Processing DeleteTablet for tablet d328d2e772ed4a6bb8597658f907f0f0 with delete_type TABLET_DATA_DELETED (replaced by tablet 827e9a16eeb24a2cadf42fbb0b12a872) from {username='slave'} at 127.0.0.1:44714
I20260807 19:42:29.747730 8479 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56494:
table { table_name: "default.Table_5d5e7f4976094d7a942e0ba75696c8f7" } modify_external_catalogs: true
I20260807 19:42:29.748032 7185 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d47a7a1285948468c0e6e1bddcd26d4 with delete_type TABLET_DATA_DELETED (replaced by tablet 541fb835769b4d1f92407b288847cdee) from {username='slave'} at 127.0.0.1:44714
I20260807 19:42:29.748082 8479 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56494:
table { table_name: "default.Table_5d5e7f4976094d7a942e0ba75696c8f7" } modify_external_catalogs: true
I20260807 19:42:29.748512 8485 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56494:
name: "default.Table_f2d1ca595d0c43ec982c1377f39155ee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:29.748804 8478 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:56494:
table { table_name: "default.Table_ca953b708a5a408898f72c483fc31ecf" } new_table_name: "default.Table_eb8c970c75a34df9a37b1812e7f5cc76" modify_external_catalogs: true
I20260807 19:42:29.749575 8477 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56494:
name: "default.Table_8461316ca98b4afab4331f54eb8d5600"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:29.749909 8583 tablet_bootstrap.cc:654] T fbea08c1f63a4a1c8a4d093be38f870e P bafdff0fd60146539631a476c4257e16: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:29.750150 8476 master_service.cc:946] ReplaceTablet: received request to replace tablet 248c1670db7244cfac99dd4817b513a4 from {username='slave'} at 127.0.0.1:56494
I20260807 19:42:29.750738 8577 tablet_bootstrap.cc:492] T 827e9a16eeb24a2cadf42fbb0b12a872 P 051d464d644a4d4486462dfb18274c6a: No bootstrap required, opened a new log
I20260807 19:42:29.751000 8577 ts_tablet_manager.cc:1397] T 827e9a16eeb24a2cadf42fbb0b12a872 P 051d464d644a4d4486462dfb18274c6a: Time spent bootstrapping tablet: real 0.024s user 0.015s sys 0.004s
I20260807 19:42:29.750896 8476 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:56494:
table { table_name: "default.Table_472c6a33c87c4f1b9cdb926a2e5a6344" } new_table_name: "default.Table_cda45f853cca4f52a3371bbaec9c7750" modify_external_catalogs: true
I20260807 19:42:29.752116 8577 raft_consensus.cc:348] T 827e9a16eeb24a2cadf42fbb0b12a872 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:42:29.752482 8577 raft_consensus.cc:374] T 827e9a16eeb24a2cadf42fbb0b12a872 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:29.752591 8577 raft_consensus.cc:729] T 827e9a16eeb24a2cadf42fbb0b12a872 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 051d464d644a4d4486462dfb18274c6a, State: Initialized, Role: FOLLOWER
I20260807 19:42:29.752727 8484 master_service.cc:946] ReplaceTablet: received request to replace tablet 9a45d0765a2c4d93ac9448496ce9c325 from {username='slave'} at 127.0.0.1:56494
I20260807 19:42:29.752933 8577 consensus_queue.cc:260] T 827e9a16eeb24a2cadf42fbb0b12a872 P 051d464d644a4d4486462dfb18274c6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:42:29.753827 8577 ts_tablet_manager.cc:1428] T 827e9a16eeb24a2cadf42fbb0b12a872 P 051d464d644a4d4486462dfb18274c6a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 19:42:29.754330 8577 tablet_bootstrap.cc:492] T f485a2f4a2a6455e8f15c73692c37dc3 P 051d464d644a4d4486462dfb18274c6a: Bootstrap starting.
I20260807 19:42:29.757912 8577 tablet_bootstrap.cc:654] T f485a2f4a2a6455e8f15c73692c37dc3 P 051d464d644a4d4486462dfb18274c6a: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:29.763501 8577 tablet_bootstrap.cc:492] T f485a2f4a2a6455e8f15c73692c37dc3 P 051d464d644a4d4486462dfb18274c6a: No bootstrap required, opened a new log
I20260807 19:42:29.763681 8577 ts_tablet_manager.cc:1397] T f485a2f4a2a6455e8f15c73692c37dc3 P 051d464d644a4d4486462dfb18274c6a: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.003s
I20260807 19:42:29.764489 8577 raft_consensus.cc:348] T f485a2f4a2a6455e8f15c73692c37dc3 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:42:29.764776 8577 raft_consensus.cc:374] T f485a2f4a2a6455e8f15c73692c37dc3 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:29.764895 8577 raft_consensus.cc:729] T f485a2f4a2a6455e8f15c73692c37dc3 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 051d464d644a4d4486462dfb18274c6a, State: Initialized, Role: FOLLOWER
I20260807 19:42:29.767562 8577 consensus_queue.cc:260] T f485a2f4a2a6455e8f15c73692c37dc3 P 051d464d644a4d4486462dfb18274c6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:42:29.768237 8577 ts_tablet_manager.cc:1428] T f485a2f4a2a6455e8f15c73692c37dc3 P 051d464d644a4d4486462dfb18274c6a: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260807 19:42:29.768558 8577 tablet_bootstrap.cc:492] T 541fb835769b4d1f92407b288847cdee P 051d464d644a4d4486462dfb18274c6a: Bootstrap starting.
I20260807 19:42:29.772856 8577 tablet_bootstrap.cc:654] T 541fb835769b4d1f92407b288847cdee P 051d464d644a4d4486462dfb18274c6a: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:29.773669 8484 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56494:
name: "default.Table_5d300b1b9ec343b88bbb7481707c963a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:29.778524 8482 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: Sending DeleteTablet for 3 replicas of tablet 769aecb6d5ca42e09edba0277a022901
I20260807 19:42:29.779583 7185 tablet_service.cc:1514] Processing DeleteTablet for tablet 769aecb6d5ca42e09edba0277a022901 with delete_type TABLET_DATA_DELETED (replaced by tablet 1b50e2d239144e2c80a169e79a235359) from {username='slave'} at 127.0.0.1:44714
I20260807 19:42:29.779768 8482 catalog_manager.cc:6565] ReplaceTablet: tablet 769aecb6d5ca42e09edba0277a022901 deleted and replaced by tablet 1b50e2d239144e2c80a169e79a235359
I20260807 19:42:29.779873 7328 tablet_service.cc:1514] Processing DeleteTablet for tablet 769aecb6d5ca42e09edba0277a022901 with delete_type TABLET_DATA_DELETED (replaced by tablet 1b50e2d239144e2c80a169e79a235359) from {username='slave'} at 127.0.0.1:51984
I20260807 19:42:29.780218 8482 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56494:
name: "default.Table_8cc3d0e85b19426f8df95f1894914926"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:29.782794 7480 tablet_service.cc:1514] Processing DeleteTablet for tablet 769aecb6d5ca42e09edba0277a022901 with delete_type TABLET_DATA_DELETED (replaced by tablet 1b50e2d239144e2c80a169e79a235359) from {username='slave'} at 127.0.0.1:33538
F20260807 19:42:29.783965 8541 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_c4966a160ae2456ebe5e58c41a9ac129
*** Check failure stack trace: ***
@ 0x7f088a5ab9ad google::LogMessage::Fail() at ??:0
I20260807 19:42:29.789115 8589 tablet_replica.cc:330] T 769aecb6d5ca42e09edba0277a022901 P bafdff0fd60146539631a476c4257e16: stopping tablet replica
I20260807 19:42:29.789479 8589 raft_consensus.cc:2229] T 769aecb6d5ca42e09edba0277a022901 P bafdff0fd60146539631a476c4257e16 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:29.790930 8589 raft_consensus.cc:2258] T 769aecb6d5ca42e09edba0277a022901 P bafdff0fd60146539631a476c4257e16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:29.791854 8589 ts_tablet_manager.cc:1905] T 769aecb6d5ca42e09edba0277a022901 P bafdff0fd60146539631a476c4257e16: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7f088a5ae88a google::LogMessage::SendToLog() at ??:0
I20260807 19:42:29.796701 7330 tablet_service.cc:1467] Processing CreateTablet for tablet 1b50e2d239144e2c80a169e79a235359 (DEFAULT_TABLE table=default.table_5d5e7f4976094d7a942e0ba75696c8f7 [id=9918b1d3805d424b92055cf267b8a4ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:29.796701 7480 tablet_service.cc:1467] Processing CreateTablet for tablet 1b50e2d239144e2c80a169e79a235359 (DEFAULT_TABLE table=default.table_5d5e7f4976094d7a942e0ba75696c8f7 [id=9918b1d3805d424b92055cf267b8a4ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:29.797201 7480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b50e2d239144e2c80a169e79a235359. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:29.797195 7330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b50e2d239144e2c80a169e79a235359. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:29.798141 7185 tablet_service.cc:1467] Processing CreateTablet for tablet 1b50e2d239144e2c80a169e79a235359 (DEFAULT_TABLE table=default.table_5d5e7f4976094d7a942e0ba75696c8f7 [id=9918b1d3805d424b92055cf267b8a4ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:29.798594 7185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b50e2d239144e2c80a169e79a235359. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7f088a5ab4de google::LogMessage::Flush() at ??:0
@ 0x7f088a5ab6d1 google::LogMessage::~LogMessage() at ??:0
I20260807 19:42:29.804654 8336 master_runner.cc:186] Error getting master registration for 127.6.109.61:39263: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111),
I20260807 19:42:29.804939 8336 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:42:29.810146 8584 tablet_replica.cc:330] T 248c1670db7244cfac99dd4817b513a4 P 051d464d644a4d4486462dfb18274c6a: stopping tablet replica
I20260807 19:42:29.810505 8584 raft_consensus.cc:2229] T 248c1670db7244cfac99dd4817b513a4 P 051d464d644a4d4486462dfb18274c6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:29.810881 8584 raft_consensus.cc:2258] T 248c1670db7244cfac99dd4817b513a4 P 051d464d644a4d4486462dfb18274c6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:29.811800 8584 ts_tablet_manager.cc:1905] T 248c1670db7244cfac99dd4817b513a4 P 051d464d644a4d4486462dfb18274c6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:29.815399 8577 tablet_bootstrap.cc:492] T 541fb835769b4d1f92407b288847cdee P 051d464d644a4d4486462dfb18274c6a: No bootstrap required, opened a new log
I20260807 19:42:29.815606 8577 ts_tablet_manager.cc:1397] T 541fb835769b4d1f92407b288847cdee P 051d464d644a4d4486462dfb18274c6a: Time spent bootstrapping tablet: real 0.047s user 0.016s sys 0.005s
I20260807 19:42:29.816444 8577 raft_consensus.cc:348] T 541fb835769b4d1f92407b288847cdee P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:42:29.816700 8577 raft_consensus.cc:374] T 541fb835769b4d1f92407b288847cdee P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:29.816785 8577 raft_consensus.cc:729] T 541fb835769b4d1f92407b288847cdee P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 051d464d644a4d4486462dfb18274c6a, State: Initialized, Role: FOLLOWER
I20260807 19:42:29.817094 8577 consensus_queue.cc:260] T 541fb835769b4d1f92407b288847cdee P 051d464d644a4d4486462dfb18274c6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:42:29.817693 8577 ts_tablet_manager.cc:1428] T 541fb835769b4d1f92407b288847cdee P 051d464d644a4d4486462dfb18274c6a: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260807 19:42:29.818038 8577 tablet_bootstrap.cc:492] T 1b50e2d239144e2c80a169e79a235359 P 051d464d644a4d4486462dfb18274c6a: Bootstrap starting.
I20260807 19:42:29.819696 8576 ts_tablet_manager.cc:1428] T f485a2f4a2a6455e8f15c73692c37dc3 P 35b5ecc2c9ef47c2bbe35653633e14da: Time spent starting tablet: real 0.080s user 0.007s sys 0.019s
I20260807 19:42:29.819875 8577 tablet_bootstrap.cc:654] T 1b50e2d239144e2c80a169e79a235359 P 051d464d644a4d4486462dfb18274c6a: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:29.820075 8576 tablet_bootstrap.cc:492] T 827e9a16eeb24a2cadf42fbb0b12a872 P 35b5ecc2c9ef47c2bbe35653633e14da: Bootstrap starting.
I20260807 19:42:29.822067 8576 tablet_bootstrap.cc:654] T 827e9a16eeb24a2cadf42fbb0b12a872 P 35b5ecc2c9ef47c2bbe35653633e14da: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:29.824623 8588 tablet_replica.cc:330] T 769aecb6d5ca42e09edba0277a022901 P 35b5ecc2c9ef47c2bbe35653633e14da: stopping tablet replica
I20260807 19:42:29.824857 8588 raft_consensus.cc:2229] T 769aecb6d5ca42e09edba0277a022901 P 35b5ecc2c9ef47c2bbe35653633e14da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:29.825158 8588 raft_consensus.cc:2258] T 769aecb6d5ca42e09edba0277a022901 P 35b5ecc2c9ef47c2bbe35653633e14da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:29.825975 8588 ts_tablet_manager.cc:1905] T 769aecb6d5ca42e09edba0277a022901 P 35b5ecc2c9ef47c2bbe35653633e14da: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7f08a0149ba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260807 19:42:29.844838 8589 ts_tablet_manager.cc:1918] T 769aecb6d5ca42e09edba0277a022901 P bafdff0fd60146539631a476c4257e16: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260807 19:42:29.845033 8589 log.cc:1196] T 769aecb6d5ca42e09edba0277a022901 P bafdff0fd60146539631a476c4257e16: Deleting WAL directory at /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-2/wal/wals/769aecb6d5ca42e09edba0277a022901
I20260807 19:42:29.845467 8589 ts_tablet_manager.cc:1939] T 769aecb6d5ca42e09edba0277a022901 P bafdff0fd60146539631a476c4257e16: Deleting consensus metadata
I20260807 19:42:29.847021 8588 ts_tablet_manager.cc:1918] T 769aecb6d5ca42e09edba0277a022901 P 35b5ecc2c9ef47c2bbe35653633e14da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260807 19:42:29.847198 8588 log.cc:1196] T 769aecb6d5ca42e09edba0277a022901 P 35b5ecc2c9ef47c2bbe35653633e14da: Deleting WAL directory at /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-1/wal/wals/769aecb6d5ca42e09edba0277a022901
I20260807 19:42:29.847697 8588 ts_tablet_manager.cc:1939] T 769aecb6d5ca42e09edba0277a022901 P 35b5ecc2c9ef47c2bbe35653633e14da: Deleting consensus metadata
I20260807 19:42:29.847872 8583 tablet_bootstrap.cc:492] T fbea08c1f63a4a1c8a4d093be38f870e P bafdff0fd60146539631a476c4257e16: No bootstrap required, opened a new log
I20260807 19:42:29.848102 8583 ts_tablet_manager.cc:1397] T fbea08c1f63a4a1c8a4d093be38f870e P bafdff0fd60146539631a476c4257e16: Time spent bootstrapping tablet: real 0.103s user 0.012s sys 0.025s
I20260807 19:42:29.849103 8472 catalog_manager.cc:4841] TS 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483): tablet 769aecb6d5ca42e09edba0277a022901 (table default.table_5d5e7f4976094d7a942e0ba75696c8f7 [id=9918b1d3805d424b92055cf267b8a4ab]) successfully deleted
I20260807 19:42:29.850147 8472 catalog_manager.cc:4841] TS bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225): tablet 769aecb6d5ca42e09edba0277a022901 (table default.table_5d5e7f4976094d7a942e0ba75696c8f7 [id=9918b1d3805d424b92055cf267b8a4ab]) successfully deleted
I20260807 19:42:29.849838 8583 raft_consensus.cc:348] T fbea08c1f63a4a1c8a4d093be38f870e P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } }
I20260807 19:42:29.850836 8583 raft_consensus.cc:374] T fbea08c1f63a4a1c8a4d093be38f870e P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:29.850996 8583 raft_consensus.cc:729] T fbea08c1f63a4a1c8a4d093be38f870e P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bafdff0fd60146539631a476c4257e16, State: Initialized, Role: FOLLOWER
I20260807 19:42:29.851025 8576 tablet_bootstrap.cc:492] T 827e9a16eeb24a2cadf42fbb0b12a872 P 35b5ecc2c9ef47c2bbe35653633e14da: No bootstrap required, opened a new log
I20260807 19:42:29.851258 8576 ts_tablet_manager.cc:1397] T 827e9a16eeb24a2cadf42fbb0b12a872 P 35b5ecc2c9ef47c2bbe35653633e14da: Time spent bootstrapping tablet: real 0.031s user 0.004s sys 0.003s
I20260807 19:42:29.851437 8583 consensus_queue.cc:260] T fbea08c1f63a4a1c8a4d093be38f870e P bafdff0fd60146539631a476c4257e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } }
I20260807 19:42:29.852212 7546 heartbeater.cc:503] Master 127.6.109.60:43129 was elected leader, sending a full tablet report...
I20260807 19:42:29.852114 8576 raft_consensus.cc:348] T 827e9a16eeb24a2cadf42fbb0b12a872 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:42:29.852286 8583 ts_tablet_manager.cc:1428] T fbea08c1f63a4a1c8a4d093be38f870e P bafdff0fd60146539631a476c4257e16: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260807 19:42:29.852381 8576 raft_consensus.cc:374] T 827e9a16eeb24a2cadf42fbb0b12a872 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:29.852428 8584 ts_tablet_manager.cc:1918] T 248c1670db7244cfac99dd4817b513a4 P 051d464d644a4d4486462dfb18274c6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260807 19:42:29.852469 8576 raft_consensus.cc:729] T 827e9a16eeb24a2cadf42fbb0b12a872 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35b5ecc2c9ef47c2bbe35653633e14da, State: Initialized, Role: FOLLOWER
I20260807 19:42:29.852658 8584 log.cc:1196] T 248c1670db7244cfac99dd4817b513a4 P 051d464d644a4d4486462dfb18274c6a: Deleting WAL directory at /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-0/wal/wals/248c1670db7244cfac99dd4817b513a4
I20260807 19:42:29.852751 8576 consensus_queue.cc:260] T 827e9a16eeb24a2cadf42fbb0b12a872 P 35b5ecc2c9ef47c2bbe35653633e14da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:42:29.852952 8583 tablet_bootstrap.cc:492] T f485a2f4a2a6455e8f15c73692c37dc3 P bafdff0fd60146539631a476c4257e16: Bootstrap starting.
I20260807 19:42:29.853361 8584 ts_tablet_manager.cc:1939] T 248c1670db7244cfac99dd4817b513a4 P 051d464d644a4d4486462dfb18274c6a: Deleting consensus metadata
I20260807 19:42:29.853423 8576 ts_tablet_manager.cc:1428] T 827e9a16eeb24a2cadf42fbb0b12a872 P 35b5ecc2c9ef47c2bbe35653633e14da: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
@ 0x7f08a0148901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260807 19:42:29.856518 8576 tablet_bootstrap.cc:492] T fbea08c1f63a4a1c8a4d093be38f870e P 35b5ecc2c9ef47c2bbe35653633e14da: Bootstrap starting.
I20260807 19:42:29.857963 8472 catalog_manager.cc:4841] TS 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): tablet 248c1670db7244cfac99dd4817b513a4 (table default.table_472c6a33c87c4f1b9cdb926a2e5a6344 [id=0b2f82ef918b47d59b34a5b5757b9486]) successfully deleted
I20260807 19:42:29.858649 8576 tablet_bootstrap.cc:654] T fbea08c1f63a4a1c8a4d093be38f870e P 35b5ecc2c9ef47c2bbe35653633e14da: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:29.860145 8584 tablet_replica.cc:330] T 9a45d0765a2c4d93ac9448496ce9c325 P 051d464d644a4d4486462dfb18274c6a: stopping tablet replica
I20260807 19:42:29.860422 8584 raft_consensus.cc:2229] T 9a45d0765a2c4d93ac9448496ce9c325 P 051d464d644a4d4486462dfb18274c6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:29.860641 8584 raft_consensus.cc:2258] T 9a45d0765a2c4d93ac9448496ce9c325 P 051d464d644a4d4486462dfb18274c6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:29.861125 8577 tablet_bootstrap.cc:492] T 1b50e2d239144e2c80a169e79a235359 P 051d464d644a4d4486462dfb18274c6a: No bootstrap required, opened a new log
I20260807 19:42:29.861255 8584 ts_tablet_manager.cc:1905] T 9a45d0765a2c4d93ac9448496ce9c325 P 051d464d644a4d4486462dfb18274c6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:29.861275 8577 ts_tablet_manager.cc:1397] T 1b50e2d239144e2c80a169e79a235359 P 051d464d644a4d4486462dfb18274c6a: Time spent bootstrapping tablet: real 0.043s user 0.024s sys 0.016s
I20260807 19:42:29.862408 8577 raft_consensus.cc:348] T 1b50e2d239144e2c80a169e79a235359 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } }
I20260807 19:42:29.862661 8577 raft_consensus.cc:374] T 1b50e2d239144e2c80a169e79a235359 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:29.862768 8577 raft_consensus.cc:729] T 1b50e2d239144e2c80a169e79a235359 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 051d464d644a4d4486462dfb18274c6a, State: Initialized, Role: FOLLOWER
I20260807 19:42:29.863030 8577 consensus_queue.cc:260] T 1b50e2d239144e2c80a169e79a235359 P 051d464d644a4d4486462dfb18274c6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } }
I20260807 19:42:29.863260 8583 tablet_bootstrap.cc:654] T f485a2f4a2a6455e8f15c73692c37dc3 P bafdff0fd60146539631a476c4257e16: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:29.863714 8577 ts_tablet_manager.cc:1428] T 1b50e2d239144e2c80a169e79a235359 P 051d464d644a4d4486462dfb18274c6a: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
@ 0x7f08a014f66c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260807 19:42:29.869040 8576 tablet_bootstrap.cc:492] T fbea08c1f63a4a1c8a4d093be38f870e P 35b5ecc2c9ef47c2bbe35653633e14da: No bootstrap required, opened a new log
I20260807 19:42:29.869244 8576 ts_tablet_manager.cc:1397] T fbea08c1f63a4a1c8a4d093be38f870e P 35b5ecc2c9ef47c2bbe35653633e14da: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.000s
@ 0x7f08a014f4e9 std::_Function_handler<>::_M_invoke() at ??:0
I20260807 19:42:29.870304 8576 raft_consensus.cc:348] T fbea08c1f63a4a1c8a4d093be38f870e P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } }
I20260807 19:42:29.870661 8576 raft_consensus.cc:374] T fbea08c1f63a4a1c8a4d093be38f870e P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:29.870770 8576 raft_consensus.cc:729] T fbea08c1f63a4a1c8a4d093be38f870e P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35b5ecc2c9ef47c2bbe35653633e14da, State: Initialized, Role: FOLLOWER
I20260807 19:42:29.871029 8583 tablet_bootstrap.cc:492] T f485a2f4a2a6455e8f15c73692c37dc3 P bafdff0fd60146539631a476c4257e16: No bootstrap required, opened a new log
I20260807 19:42:29.871236 8583 ts_tablet_manager.cc:1397] T f485a2f4a2a6455e8f15c73692c37dc3 P bafdff0fd60146539631a476c4257e16: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.000s
I20260807 19:42:29.871138 8576 consensus_queue.cc:260] T fbea08c1f63a4a1c8a4d093be38f870e P 35b5ecc2c9ef47c2bbe35653633e14da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } }
@ 0x9edb7d std::function<>::operator()() at ??:?
I20260807 19:42:29.872130 8583 raft_consensus.cc:348] T f485a2f4a2a6455e8f15c73692c37dc3 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:42:29.872394 8583 raft_consensus.cc:374] T f485a2f4a2a6455e8f15c73692c37dc3 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:29.872476 8583 raft_consensus.cc:729] T f485a2f4a2a6455e8f15c73692c37dc3 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bafdff0fd60146539631a476c4257e16, State: Initialized, Role: FOLLOWER
I20260807 19:42:29.871394 8584 ts_tablet_manager.cc:1918] T 9a45d0765a2c4d93ac9448496ce9c325 P 051d464d644a4d4486462dfb18274c6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260807 19:42:29.872731 8583 consensus_queue.cc:260] T f485a2f4a2a6455e8f15c73692c37dc3 P bafdff0fd60146539631a476c4257e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:42:29.873032 8584 log.cc:1196] T 9a45d0765a2c4d93ac9448496ce9c325 P 051d464d644a4d4486462dfb18274c6a: Deleting WAL directory at /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-0/wal/wals/9a45d0765a2c4d93ac9448496ce9c325
I20260807 19:42:29.873520 8583 ts_tablet_manager.cc:1428] T f485a2f4a2a6455e8f15c73692c37dc3 P bafdff0fd60146539631a476c4257e16: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260807 19:42:29.873860 8583 tablet_bootstrap.cc:492] T 541fb835769b4d1f92407b288847cdee P bafdff0fd60146539631a476c4257e16: Bootstrap starting.
I20260807 19:42:29.875656 8584 ts_tablet_manager.cc:1939] T 9a45d0765a2c4d93ac9448496ce9c325 P 051d464d644a4d4486462dfb18274c6a: Deleting consensus metadata
I20260807 19:42:29.875794 8583 tablet_bootstrap.cc:654] T 541fb835769b4d1f92407b288847cdee P bafdff0fd60146539631a476c4257e16: Neither blocks nor log segments found. Creating new log.
@ 0x7f088d1d5eaf kudu::Thread::SuperviseThread() at ??:0
I20260807 19:42:29.876992 8472 catalog_manager.cc:4841] TS 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): tablet 9a45d0765a2c4d93ac9448496ce9c325 (table default.table_49a9f7854b4c4847bce27e16296322b5 [id=cf4da31cb40447199dc079b8b6e709bc]) successfully deleted
I20260807 19:42:29.877522 8584 tablet_replica.cc:330] T d328d2e772ed4a6bb8597658f907f0f0 P 051d464d644a4d4486462dfb18274c6a: stopping tablet replica
I20260807 19:42:29.877540 8576 ts_tablet_manager.cc:1428] T fbea08c1f63a4a1c8a4d093be38f870e P 35b5ecc2c9ef47c2bbe35653633e14da: Time spent starting tablet: real 0.008s user 0.003s sys 0.000s
I20260807 19:42:29.877827 8584 raft_consensus.cc:2229] T d328d2e772ed4a6bb8597658f907f0f0 P 051d464d644a4d4486462dfb18274c6a [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:42:29.878541 8584 raft_consensus.cc:2258] T d328d2e772ed4a6bb8597658f907f0f0 P 051d464d644a4d4486462dfb18274c6a [term 1 FOLLOWER]: Raft consensus is shut down!
@ 0x7f08904af6db start_thread at ??:0
I20260807 19:42:29.879349 8584 ts_tablet_manager.cc:1905] T d328d2e772ed4a6bb8597658f907f0f0 P 051d464d644a4d4486462dfb18274c6a: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7f088944561f clone at ??:0
I20260807 19:42:29.881546 8576 tablet_bootstrap.cc:492] T 541fb835769b4d1f92407b288847cdee P 35b5ecc2c9ef47c2bbe35653633e14da: Bootstrap starting.
I20260807 19:42:29.883757 8576 tablet_bootstrap.cc:654] T 541fb835769b4d1f92407b288847cdee P 35b5ecc2c9ef47c2bbe35653633e14da: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:29.888710 8584 ts_tablet_manager.cc:1918] T d328d2e772ed4a6bb8597658f907f0f0 P 051d464d644a4d4486462dfb18274c6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260807 19:42:29.888865 8584 log.cc:1196] T d328d2e772ed4a6bb8597658f907f0f0 P 051d464d644a4d4486462dfb18274c6a: Deleting WAL directory at /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-0/wal/wals/d328d2e772ed4a6bb8597658f907f0f0
I20260807 19:42:29.888885 8583 tablet_bootstrap.cc:492] T 541fb835769b4d1f92407b288847cdee P bafdff0fd60146539631a476c4257e16: No bootstrap required, opened a new log
I20260807 19:42:29.889050 8583 ts_tablet_manager.cc:1397] T 541fb835769b4d1f92407b288847cdee P bafdff0fd60146539631a476c4257e16: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.007s
I20260807 19:42:29.889380 8584 ts_tablet_manager.cc:1939] T d328d2e772ed4a6bb8597658f907f0f0 P 051d464d644a4d4486462dfb18274c6a: Deleting consensus metadata
I20260807 19:42:29.889945 8576 tablet_bootstrap.cc:492] T 541fb835769b4d1f92407b288847cdee P 35b5ecc2c9ef47c2bbe35653633e14da: No bootstrap required, opened a new log
I20260807 19:42:29.890136 8576 ts_tablet_manager.cc:1397] T 541fb835769b4d1f92407b288847cdee P 35b5ecc2c9ef47c2bbe35653633e14da: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.002s
I20260807 19:42:29.890080 8583 raft_consensus.cc:348] T 541fb835769b4d1f92407b288847cdee P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:42:29.890345 8583 raft_consensus.cc:374] T 541fb835769b4d1f92407b288847cdee P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:29.890455 8583 raft_consensus.cc:729] T 541fb835769b4d1f92407b288847cdee P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bafdff0fd60146539631a476c4257e16, State: Initialized, Role: FOLLOWER
I20260807 19:42:29.890674 8584 tablet_replica.cc:330] T 0d47a7a1285948468c0e6e1bddcd26d4 P 051d464d644a4d4486462dfb18274c6a: stopping tablet replica
I20260807 19:42:29.890767 8583 consensus_queue.cc:260] T 541fb835769b4d1f92407b288847cdee P bafdff0fd60146539631a476c4257e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:42:29.890892 8584 raft_consensus.cc:2229] T 0d47a7a1285948468c0e6e1bddcd26d4 P 051d464d644a4d4486462dfb18274c6a [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:42:29.890940 8576 raft_consensus.cc:348] T 541fb835769b4d1f92407b288847cdee P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:42:29.891201 8576 raft_consensus.cc:374] T 541fb835769b4d1f92407b288847cdee P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:29.891271 8576 raft_consensus.cc:729] T 541fb835769b4d1f92407b288847cdee P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35b5ecc2c9ef47c2bbe35653633e14da, State: Initialized, Role: FOLLOWER
I20260807 19:42:29.891544 8576 consensus_queue.cc:260] T 541fb835769b4d1f92407b288847cdee P 35b5ecc2c9ef47c2bbe35653633e14da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:42:29.892575 8583 ts_tablet_manager.cc:1428] T 541fb835769b4d1f92407b288847cdee P bafdff0fd60146539631a476c4257e16: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 19:42:29.892907 8583 tablet_bootstrap.cc:492] T 827e9a16eeb24a2cadf42fbb0b12a872 P bafdff0fd60146539631a476c4257e16: Bootstrap starting.
I20260807 19:42:29.893940 8584 raft_consensus.cc:2258] T 0d47a7a1285948468c0e6e1bddcd26d4 P 051d464d644a4d4486462dfb18274c6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:29.894670 8584 ts_tablet_manager.cc:1905] T 0d47a7a1285948468c0e6e1bddcd26d4 P 051d464d644a4d4486462dfb18274c6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:29.894894 8583 tablet_bootstrap.cc:654] T 827e9a16eeb24a2cadf42fbb0b12a872 P bafdff0fd60146539631a476c4257e16: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:29.897184 8583 tablet_bootstrap.cc:492] T 827e9a16eeb24a2cadf42fbb0b12a872 P bafdff0fd60146539631a476c4257e16: No bootstrap required, opened a new log
I20260807 19:42:29.897347 8583 ts_tablet_manager.cc:1397] T 827e9a16eeb24a2cadf42fbb0b12a872 P bafdff0fd60146539631a476c4257e16: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260807 19:42:29.898164 8583 raft_consensus.cc:348] T 827e9a16eeb24a2cadf42fbb0b12a872 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:42:29.898434 8583 raft_consensus.cc:374] T 827e9a16eeb24a2cadf42fbb0b12a872 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:29.898520 8583 raft_consensus.cc:729] T 827e9a16eeb24a2cadf42fbb0b12a872 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bafdff0fd60146539631a476c4257e16, State: Initialized, Role: FOLLOWER
I20260807 19:42:29.898793 8583 consensus_queue.cc:260] T 827e9a16eeb24a2cadf42fbb0b12a872 P bafdff0fd60146539631a476c4257e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:42:29.899566 8583 ts_tablet_manager.cc:1428] T 827e9a16eeb24a2cadf42fbb0b12a872 P bafdff0fd60146539631a476c4257e16: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 19:42:29.899904 8583 tablet_bootstrap.cc:492] T 1b50e2d239144e2c80a169e79a235359 P bafdff0fd60146539631a476c4257e16: Bootstrap starting.
I20260807 19:42:29.901854 8583 tablet_bootstrap.cc:654] T 1b50e2d239144e2c80a169e79a235359 P bafdff0fd60146539631a476c4257e16: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:29.902189 8576 ts_tablet_manager.cc:1428] T 541fb835769b4d1f92407b288847cdee P 35b5ecc2c9ef47c2bbe35653633e14da: Time spent starting tablet: real 0.012s user 0.002s sys 0.000s
I20260807 19:42:29.906052 8576 tablet_bootstrap.cc:492] T 1b50e2d239144e2c80a169e79a235359 P 35b5ecc2c9ef47c2bbe35653633e14da: Bootstrap starting.
I20260807 19:42:29.908484 8584 ts_tablet_manager.cc:1918] T 0d47a7a1285948468c0e6e1bddcd26d4 P 051d464d644a4d4486462dfb18274c6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260807 19:42:29.908708 8584 log.cc:1196] T 0d47a7a1285948468c0e6e1bddcd26d4 P 051d464d644a4d4486462dfb18274c6a: Deleting WAL directory at /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-0/wal/wals/0d47a7a1285948468c0e6e1bddcd26d4
I20260807 19:42:29.909241 8584 ts_tablet_manager.cc:1939] T 0d47a7a1285948468c0e6e1bddcd26d4 P 051d464d644a4d4486462dfb18274c6a: Deleting consensus metadata
I20260807 19:42:29.910012 8576 tablet_bootstrap.cc:654] T 1b50e2d239144e2c80a169e79a235359 P 35b5ecc2c9ef47c2bbe35653633e14da: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:29.915012 8584 tablet_replica.cc:330] T 769aecb6d5ca42e09edba0277a022901 P 051d464d644a4d4486462dfb18274c6a: stopping tablet replica
I20260807 19:42:29.915343 8584 raft_consensus.cc:2229] T 769aecb6d5ca42e09edba0277a022901 P 051d464d644a4d4486462dfb18274c6a [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:42:29.915824 8584 raft_consensus.cc:2258] T 769aecb6d5ca42e09edba0277a022901 P 051d464d644a4d4486462dfb18274c6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:29.916680 8584 ts_tablet_manager.cc:1905] T 769aecb6d5ca42e09edba0277a022901 P 051d464d644a4d4486462dfb18274c6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:29.923180 8583 tablet_bootstrap.cc:492] T 1b50e2d239144e2c80a169e79a235359 P bafdff0fd60146539631a476c4257e16: No bootstrap required, opened a new log
I20260807 19:42:29.923380 8583 ts_tablet_manager.cc:1397] T 1b50e2d239144e2c80a169e79a235359 P bafdff0fd60146539631a476c4257e16: Time spent bootstrapping tablet: real 0.024s user 0.004s sys 0.019s
I20260807 19:42:29.924347 8583 raft_consensus.cc:348] T 1b50e2d239144e2c80a169e79a235359 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } }
I20260807 19:42:29.924625 8583 raft_consensus.cc:374] T 1b50e2d239144e2c80a169e79a235359 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:29.924723 8583 raft_consensus.cc:729] T 1b50e2d239144e2c80a169e79a235359 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bafdff0fd60146539631a476c4257e16, State: Initialized, Role: FOLLOWER
I20260807 19:42:29.925082 8583 consensus_queue.cc:260] T 1b50e2d239144e2c80a169e79a235359 P bafdff0fd60146539631a476c4257e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } }
I20260807 19:42:29.926559 8583 ts_tablet_manager.cc:1428] T 1b50e2d239144e2c80a169e79a235359 P bafdff0fd60146539631a476c4257e16: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260807 19:42:29.931370 8576 tablet_bootstrap.cc:492] T 1b50e2d239144e2c80a169e79a235359 P 35b5ecc2c9ef47c2bbe35653633e14da: No bootstrap required, opened a new log
I20260807 19:42:29.931550 8576 ts_tablet_manager.cc:1397] T 1b50e2d239144e2c80a169e79a235359 P 35b5ecc2c9ef47c2bbe35653633e14da: Time spent bootstrapping tablet: real 0.026s user 0.005s sys 0.012s
I20260807 19:42:29.931928 8584 ts_tablet_manager.cc:1918] T 769aecb6d5ca42e09edba0277a022901 P 051d464d644a4d4486462dfb18274c6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260807 19:42:29.932503 8584 log.cc:1196] T 769aecb6d5ca42e09edba0277a022901 P 051d464d644a4d4486462dfb18274c6a: Deleting WAL directory at /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-0/wal/wals/769aecb6d5ca42e09edba0277a022901
I20260807 19:42:29.932428 8576 raft_consensus.cc:348] T 1b50e2d239144e2c80a169e79a235359 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } }
I20260807 19:42:29.932682 8576 raft_consensus.cc:374] T 1b50e2d239144e2c80a169e79a235359 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:29.932770 8576 raft_consensus.cc:729] T 1b50e2d239144e2c80a169e79a235359 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35b5ecc2c9ef47c2bbe35653633e14da, State: Initialized, Role: FOLLOWER
I20260807 19:42:29.932967 8584 ts_tablet_manager.cc:1939] T 769aecb6d5ca42e09edba0277a022901 P 051d464d644a4d4486462dfb18274c6a: Deleting consensus metadata
I20260807 19:42:29.933077 8576 consensus_queue.cc:260] T 1b50e2d239144e2c80a169e79a235359 P 35b5ecc2c9ef47c2bbe35653633e14da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } }
I20260807 19:42:29.933795 8576 ts_tablet_manager.cc:1428] T 1b50e2d239144e2c80a169e79a235359 P 35b5ecc2c9ef47c2bbe35653633e14da: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260807 19:42:29.984673 8320 raft_consensus.cc:482] T fbea08c1f63a4a1c8a4d093be38f870e P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:29.984947 8320 raft_consensus.cc:504] T fbea08c1f63a4a1c8a4d093be38f870e P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } }
I20260807 19:42:29.986078 8320 leader_election.cc:288] T fbea08c1f63a4a1c8a4d093be38f870e P 051d464d644a4d4486462dfb18274c6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225), 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483)
I20260807 19:42:29.987871 7351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbea08c1f63a4a1c8a4d093be38f870e" candidate_uuid: "051d464d644a4d4486462dfb18274c6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" is_pre_election: true
I20260807 19:42:29.988233 7351 raft_consensus.cc:2454] T fbea08c1f63a4a1c8a4d093be38f870e P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 051d464d644a4d4486462dfb18274c6a in term 0.
I20260807 19:42:29.988322 7501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbea08c1f63a4a1c8a4d093be38f870e" candidate_uuid: "051d464d644a4d4486462dfb18274c6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bafdff0fd60146539631a476c4257e16" is_pre_election: true
I20260807 19:42:29.988622 7501 raft_consensus.cc:2454] T fbea08c1f63a4a1c8a4d093be38f870e P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 051d464d644a4d4486462dfb18274c6a in term 0.
I20260807 19:42:29.988870 7140 leader_election.cc:302] T fbea08c1f63a4a1c8a4d093be38f870e P 051d464d644a4d4486462dfb18274c6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051d464d644a4d4486462dfb18274c6a, 35b5ecc2c9ef47c2bbe35653633e14da; no voters:
I20260807 19:42:29.989432 8320 raft_consensus.cc:2781] T fbea08c1f63a4a1c8a4d093be38f870e P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:42:29.989569 8320 raft_consensus.cc:482] T fbea08c1f63a4a1c8a4d093be38f870e P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:42:29.989705 8320 raft_consensus.cc:3037] T fbea08c1f63a4a1c8a4d093be38f870e P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:29.992619 8320 raft_consensus.cc:504] T fbea08c1f63a4a1c8a4d093be38f870e P 051d464d644a4d4486462dfb18274c6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } }
I20260807 19:42:29.993352 8320 leader_election.cc:288] T fbea08c1f63a4a1c8a4d093be38f870e P 051d464d644a4d4486462dfb18274c6a [CANDIDATE]: Term 1 election: Requested vote from peers bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225), 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483)
I20260807 19:42:29.994005 7501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbea08c1f63a4a1c8a4d093be38f870e" candidate_uuid: "051d464d644a4d4486462dfb18274c6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bafdff0fd60146539631a476c4257e16"
I20260807 19:42:29.994088 7351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbea08c1f63a4a1c8a4d093be38f870e" candidate_uuid: "051d464d644a4d4486462dfb18274c6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35b5ecc2c9ef47c2bbe35653633e14da"
I20260807 19:42:29.994319 7501 raft_consensus.cc:3037] T fbea08c1f63a4a1c8a4d093be38f870e P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:29.994365 7351 raft_consensus.cc:3037] T fbea08c1f63a4a1c8a4d093be38f870e P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:29.997175 7351 raft_consensus.cc:2454] T fbea08c1f63a4a1c8a4d093be38f870e P 35b5ecc2c9ef47c2bbe35653633e14da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 051d464d644a4d4486462dfb18274c6a in term 1.
I20260807 19:42:29.997303 7501 raft_consensus.cc:2454] T fbea08c1f63a4a1c8a4d093be38f870e P bafdff0fd60146539631a476c4257e16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 051d464d644a4d4486462dfb18274c6a in term 1.
I20260807 19:42:29.997666 8593 raft_consensus.cc:482] T 1b50e2d239144e2c80a169e79a235359 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:29.997960 7140 leader_election.cc:302] T fbea08c1f63a4a1c8a4d093be38f870e P 051d464d644a4d4486462dfb18274c6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051d464d644a4d4486462dfb18274c6a, 35b5ecc2c9ef47c2bbe35653633e14da; no voters:
I20260807 19:42:29.997856 8593 raft_consensus.cc:504] T 1b50e2d239144e2c80a169e79a235359 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } }
I20260807 19:42:29.998369 8320 raft_consensus.cc:2781] T fbea08c1f63a4a1c8a4d093be38f870e P 051d464d644a4d4486462dfb18274c6a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:29.998935 8593 leader_election.cc:288] T 1b50e2d239144e2c80a169e79a235359 P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225), 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467)
I20260807 19:42:29.999013 8320 raft_consensus.cc:686] T fbea08c1f63a4a1c8a4d093be38f870e P 051d464d644a4d4486462dfb18274c6a [term 1 LEADER]: Becoming Leader. State: Replica: 051d464d644a4d4486462dfb18274c6a, State: Running, Role: LEADER
I20260807 19:42:29.999351 8320 consensus_queue.cc:237] T fbea08c1f63a4a1c8a4d093be38f870e P 051d464d644a4d4486462dfb18274c6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } }
I20260807 19:42:30.000712 7208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b50e2d239144e2c80a169e79a235359" candidate_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051d464d644a4d4486462dfb18274c6a" is_pre_election: true
I20260807 19:42:30.000746 7501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b50e2d239144e2c80a169e79a235359" candidate_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bafdff0fd60146539631a476c4257e16" is_pre_election: true
I20260807 19:42:30.000923 7208 raft_consensus.cc:2454] T 1b50e2d239144e2c80a169e79a235359 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35b5ecc2c9ef47c2bbe35653633e14da in term 0.
I20260807 19:42:30.001061 7501 raft_consensus.cc:2454] T 1b50e2d239144e2c80a169e79a235359 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35b5ecc2c9ef47c2bbe35653633e14da in term 0.
I20260807 19:42:30.001399 7287 leader_election.cc:302] T 1b50e2d239144e2c80a169e79a235359 P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051d464d644a4d4486462dfb18274c6a, 35b5ecc2c9ef47c2bbe35653633e14da; no voters:
I20260807 19:42:30.001883 8593 raft_consensus.cc:2781] T 1b50e2d239144e2c80a169e79a235359 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:42:30.002017 8593 raft_consensus.cc:482] T 1b50e2d239144e2c80a169e79a235359 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:42:30.002125 8593 raft_consensus.cc:3037] T 1b50e2d239144e2c80a169e79a235359 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:30.005251 8593 raft_consensus.cc:504] T 1b50e2d239144e2c80a169e79a235359 P 35b5ecc2c9ef47c2bbe35653633e14da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } }
I20260807 19:42:30.006068 8593 leader_election.cc:288] T 1b50e2d239144e2c80a169e79a235359 P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 election: Requested vote from peers bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225), 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467)
I20260807 19:42:30.006659 7501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b50e2d239144e2c80a169e79a235359" candidate_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bafdff0fd60146539631a476c4257e16"
I20260807 19:42:30.006834 7208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b50e2d239144e2c80a169e79a235359" candidate_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051d464d644a4d4486462dfb18274c6a"
I20260807 19:42:30.006937 7501 raft_consensus.cc:3037] T 1b50e2d239144e2c80a169e79a235359 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:30.007086 7208 raft_consensus.cc:3037] T 1b50e2d239144e2c80a169e79a235359 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:30.010079 7501 raft_consensus.cc:2454] T 1b50e2d239144e2c80a169e79a235359 P bafdff0fd60146539631a476c4257e16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35b5ecc2c9ef47c2bbe35653633e14da in term 1.
I20260807 19:42:30.010078 7208 raft_consensus.cc:2454] T 1b50e2d239144e2c80a169e79a235359 P 051d464d644a4d4486462dfb18274c6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35b5ecc2c9ef47c2bbe35653633e14da in term 1.
I20260807 19:42:30.010656 7287 leader_election.cc:302] T 1b50e2d239144e2c80a169e79a235359 P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35b5ecc2c9ef47c2bbe35653633e14da, bafdff0fd60146539631a476c4257e16; no voters:
I20260807 19:42:30.011044 8593 raft_consensus.cc:2781] T 1b50e2d239144e2c80a169e79a235359 P 35b5ecc2c9ef47c2bbe35653633e14da [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:30.011245 8593 raft_consensus.cc:686] T 1b50e2d239144e2c80a169e79a235359 P 35b5ecc2c9ef47c2bbe35653633e14da [term 1 LEADER]: Becoming Leader. State: Replica: 35b5ecc2c9ef47c2bbe35653633e14da, State: Running, Role: LEADER
I20260807 19:42:30.011582 8593 consensus_queue.cc:237] T 1b50e2d239144e2c80a169e79a235359 P 35b5ecc2c9ef47c2bbe35653633e14da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } }
I20260807 19:42:30.042994 8320 raft_consensus.cc:482] T 827e9a16eeb24a2cadf42fbb0b12a872 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:30.043294 8320 raft_consensus.cc:504] T 827e9a16eeb24a2cadf42fbb0b12a872 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:42:30.044157 8320 leader_election.cc:288] T 827e9a16eeb24a2cadf42fbb0b12a872 P 051d464d644a4d4486462dfb18274c6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225), 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483)
I20260807 19:42:30.044971 7351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "827e9a16eeb24a2cadf42fbb0b12a872" candidate_uuid: "051d464d644a4d4486462dfb18274c6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" is_pre_election: true
I20260807 19:42:30.044981 7501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "827e9a16eeb24a2cadf42fbb0b12a872" candidate_uuid: "051d464d644a4d4486462dfb18274c6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bafdff0fd60146539631a476c4257e16" is_pre_election: true
I20260807 19:42:30.045300 7351 raft_consensus.cc:2454] T 827e9a16eeb24a2cadf42fbb0b12a872 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 051d464d644a4d4486462dfb18274c6a in term 0.
I20260807 19:42:30.045300 7501 raft_consensus.cc:2454] T 827e9a16eeb24a2cadf42fbb0b12a872 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 051d464d644a4d4486462dfb18274c6a in term 0.
I20260807 19:42:30.045866 7140 leader_election.cc:302] T 827e9a16eeb24a2cadf42fbb0b12a872 P 051d464d644a4d4486462dfb18274c6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051d464d644a4d4486462dfb18274c6a, 35b5ecc2c9ef47c2bbe35653633e14da; no voters:
I20260807 19:42:30.046232 8320 raft_consensus.cc:2781] T 827e9a16eeb24a2cadf42fbb0b12a872 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:42:30.046352 8320 raft_consensus.cc:482] T 827e9a16eeb24a2cadf42fbb0b12a872 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:42:30.046469 8320 raft_consensus.cc:3037] T 827e9a16eeb24a2cadf42fbb0b12a872 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:30.049492 8320 raft_consensus.cc:504] T 827e9a16eeb24a2cadf42fbb0b12a872 P 051d464d644a4d4486462dfb18274c6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:42:30.050343 8320 leader_election.cc:288] T 827e9a16eeb24a2cadf42fbb0b12a872 P 051d464d644a4d4486462dfb18274c6a [CANDIDATE]: Term 1 election: Requested vote from peers bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225), 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483)
I20260807 19:42:30.051182 7501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "827e9a16eeb24a2cadf42fbb0b12a872" candidate_uuid: "051d464d644a4d4486462dfb18274c6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bafdff0fd60146539631a476c4257e16"
I20260807 19:42:30.051332 7351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "827e9a16eeb24a2cadf42fbb0b12a872" candidate_uuid: "051d464d644a4d4486462dfb18274c6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35b5ecc2c9ef47c2bbe35653633e14da"
I20260807 19:42:30.051486 7501 raft_consensus.cc:3037] T 827e9a16eeb24a2cadf42fbb0b12a872 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:30.051610 7351 raft_consensus.cc:3037] T 827e9a16eeb24a2cadf42fbb0b12a872 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:30.054559 7351 raft_consensus.cc:2454] T 827e9a16eeb24a2cadf42fbb0b12a872 P 35b5ecc2c9ef47c2bbe35653633e14da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 051d464d644a4d4486462dfb18274c6a in term 1.
I20260807 19:42:30.054558 7501 raft_consensus.cc:2454] T 827e9a16eeb24a2cadf42fbb0b12a872 P bafdff0fd60146539631a476c4257e16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 051d464d644a4d4486462dfb18274c6a in term 1.
I20260807 19:42:30.055120 7140 leader_election.cc:302] T 827e9a16eeb24a2cadf42fbb0b12a872 P 051d464d644a4d4486462dfb18274c6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051d464d644a4d4486462dfb18274c6a, bafdff0fd60146539631a476c4257e16; no voters:
I20260807 19:42:30.055570 8320 raft_consensus.cc:2781] T 827e9a16eeb24a2cadf42fbb0b12a872 P 051d464d644a4d4486462dfb18274c6a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:30.055783 8320 raft_consensus.cc:686] T 827e9a16eeb24a2cadf42fbb0b12a872 P 051d464d644a4d4486462dfb18274c6a [term 1 LEADER]: Becoming Leader. State: Replica: 051d464d644a4d4486462dfb18274c6a, State: Running, Role: LEADER
I20260807 19:42:30.056171 8320 consensus_queue.cc:237] T 827e9a16eeb24a2cadf42fbb0b12a872 P 051d464d644a4d4486462dfb18274c6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
W20260807 19:42:30.104966 7140 connection.cc:535] server connection from 127.0.0.1:44714 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 19:42:30.105676 7559 connection.cc:535] client connection to 127.6.109.60:43129 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 19:42:30.106922 7559 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.6.109.60:43129)
W20260807 19:42:30.107717 7559 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.6.109.60:43129)
W20260807 19:42:30.108189 7559 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.6.109.60:43129)
W20260807 19:42:30.108469 7559 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.6.109.60:43129)
I20260807 19:42:30.238721 8547 raft_consensus.cc:482] T 541fb835769b4d1f92407b288847cdee P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:30.238983 8547 raft_consensus.cc:504] T 541fb835769b4d1f92407b288847cdee P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:42:30.239984 8547 leader_election.cc:288] T 541fb835769b4d1f92407b288847cdee P bafdff0fd60146539631a476c4257e16 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467), 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483)
I20260807 19:42:30.240664 7208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "541fb835769b4d1f92407b288847cdee" candidate_uuid: "bafdff0fd60146539631a476c4257e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051d464d644a4d4486462dfb18274c6a" is_pre_election: true
I20260807 19:42:30.240792 7351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "541fb835769b4d1f92407b288847cdee" candidate_uuid: "bafdff0fd60146539631a476c4257e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" is_pre_election: true
I20260807 19:42:30.240984 7208 raft_consensus.cc:2454] T 541fb835769b4d1f92407b288847cdee P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bafdff0fd60146539631a476c4257e16 in term 0.
I20260807 19:42:30.241111 7351 raft_consensus.cc:2454] T 541fb835769b4d1f92407b288847cdee P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bafdff0fd60146539631a476c4257e16 in term 0.
I20260807 19:42:30.241515 7436 leader_election.cc:302] T 541fb835769b4d1f92407b288847cdee P bafdff0fd60146539631a476c4257e16 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051d464d644a4d4486462dfb18274c6a, bafdff0fd60146539631a476c4257e16; no voters:
I20260807 19:42:30.241865 8547 raft_consensus.cc:2781] T 541fb835769b4d1f92407b288847cdee P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:42:30.242003 8547 raft_consensus.cc:482] T 541fb835769b4d1f92407b288847cdee P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:42:30.242100 8547 raft_consensus.cc:3037] T 541fb835769b4d1f92407b288847cdee P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:30.244308 8547 raft_consensus.cc:504] T 541fb835769b4d1f92407b288847cdee P bafdff0fd60146539631a476c4257e16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:42:30.244920 8547 leader_election.cc:288] T 541fb835769b4d1f92407b288847cdee P bafdff0fd60146539631a476c4257e16 [CANDIDATE]: Term 1 election: Requested vote from peers 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467), 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483)
I20260807 19:42:30.245541 7208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "541fb835769b4d1f92407b288847cdee" candidate_uuid: "bafdff0fd60146539631a476c4257e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051d464d644a4d4486462dfb18274c6a"
I20260807 19:42:30.245826 7208 raft_consensus.cc:3037] T 541fb835769b4d1f92407b288847cdee P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:30.246130 7351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "541fb835769b4d1f92407b288847cdee" candidate_uuid: "bafdff0fd60146539631a476c4257e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35b5ecc2c9ef47c2bbe35653633e14da"
I20260807 19:42:30.246353 7351 raft_consensus.cc:3037] T 541fb835769b4d1f92407b288847cdee P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:30.248006 7208 raft_consensus.cc:2454] T 541fb835769b4d1f92407b288847cdee P 051d464d644a4d4486462dfb18274c6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bafdff0fd60146539631a476c4257e16 in term 1.
I20260807 19:42:30.248535 7436 leader_election.cc:302] T 541fb835769b4d1f92407b288847cdee P bafdff0fd60146539631a476c4257e16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051d464d644a4d4486462dfb18274c6a, bafdff0fd60146539631a476c4257e16; no voters:
I20260807 19:42:30.248696 7351 raft_consensus.cc:2454] T 541fb835769b4d1f92407b288847cdee P 35b5ecc2c9ef47c2bbe35653633e14da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bafdff0fd60146539631a476c4257e16 in term 1.
I20260807 19:42:30.248868 8547 raft_consensus.cc:2781] T 541fb835769b4d1f92407b288847cdee P bafdff0fd60146539631a476c4257e16 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:30.249879 8547 raft_consensus.cc:686] T 541fb835769b4d1f92407b288847cdee P bafdff0fd60146539631a476c4257e16 [term 1 LEADER]: Becoming Leader. State: Replica: bafdff0fd60146539631a476c4257e16, State: Running, Role: LEADER
I20260807 19:42:30.250218 8547 consensus_queue.cc:237] T 541fb835769b4d1f92407b288847cdee P bafdff0fd60146539631a476c4257e16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:42:30.295763 8619 raft_consensus.cc:482] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7b31edda0c4a48489ad60fc59c44ed31)
I20260807 19:42:30.296030 8619 raft_consensus.cc:504] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:30.296746 8619 leader_election.cc:288] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263), 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129)
W20260807 19:42:30.298899 8354 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:42:30.299093 8354 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129): Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: connect: Connection refused (error 111)
I20260807 19:42:30.299160 8354 leader_election.cc:302] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 959ac7807a8b42a59c61a23d4605c054; no voters: 1b782e30a8d44e7b94f080aa6327134f, 7b31edda0c4a48489ad60fc59c44ed31
I20260807 19:42:30.299401 8619 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260807 19:42:30.307585 7436 consensus_peers.cc:487] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 248c1670db7244cfac99dd4817b513a4. This is attempt 1: this message will repeat every 5th retry.
I20260807 19:42:30.413583 8320 raft_consensus.cc:482] T f485a2f4a2a6455e8f15c73692c37dc3 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:30.413869 8320 raft_consensus.cc:504] T f485a2f4a2a6455e8f15c73692c37dc3 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:42:30.414464 8320 leader_election.cc:288] T f485a2f4a2a6455e8f15c73692c37dc3 P 051d464d644a4d4486462dfb18274c6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225), 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483)
I20260807 19:42:30.415189 7501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f485a2f4a2a6455e8f15c73692c37dc3" candidate_uuid: "051d464d644a4d4486462dfb18274c6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bafdff0fd60146539631a476c4257e16" is_pre_election: true
I20260807 19:42:30.415284 7351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f485a2f4a2a6455e8f15c73692c37dc3" candidate_uuid: "051d464d644a4d4486462dfb18274c6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" is_pre_election: true
I20260807 19:42:30.415467 7501 raft_consensus.cc:2454] T f485a2f4a2a6455e8f15c73692c37dc3 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 051d464d644a4d4486462dfb18274c6a in term 0.
I20260807 19:42:30.415620 7351 raft_consensus.cc:2454] T f485a2f4a2a6455e8f15c73692c37dc3 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 051d464d644a4d4486462dfb18274c6a in term 0.
I20260807 19:42:30.416023 7140 leader_election.cc:302] T f485a2f4a2a6455e8f15c73692c37dc3 P 051d464d644a4d4486462dfb18274c6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051d464d644a4d4486462dfb18274c6a, bafdff0fd60146539631a476c4257e16; no voters:
I20260807 19:42:30.416323 8320 raft_consensus.cc:2781] T f485a2f4a2a6455e8f15c73692c37dc3 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:42:30.416476 8320 raft_consensus.cc:482] T f485a2f4a2a6455e8f15c73692c37dc3 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:42:30.416574 8320 raft_consensus.cc:3037] T f485a2f4a2a6455e8f15c73692c37dc3 P 051d464d644a4d4486462dfb18274c6a [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:30.418563 8320 raft_consensus.cc:504] T f485a2f4a2a6455e8f15c73692c37dc3 P 051d464d644a4d4486462dfb18274c6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:42:30.419118 8320 leader_election.cc:288] T f485a2f4a2a6455e8f15c73692c37dc3 P 051d464d644a4d4486462dfb18274c6a [CANDIDATE]: Term 1 election: Requested vote from peers bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225), 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483)
I20260807 19:42:30.419754 7501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f485a2f4a2a6455e8f15c73692c37dc3" candidate_uuid: "051d464d644a4d4486462dfb18274c6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bafdff0fd60146539631a476c4257e16"
I20260807 19:42:30.419727 7351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f485a2f4a2a6455e8f15c73692c37dc3" candidate_uuid: "051d464d644a4d4486462dfb18274c6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35b5ecc2c9ef47c2bbe35653633e14da"
I20260807 19:42:30.419996 7501 raft_consensus.cc:3037] T f485a2f4a2a6455e8f15c73692c37dc3 P bafdff0fd60146539631a476c4257e16 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:30.420060 7351 raft_consensus.cc:3037] T f485a2f4a2a6455e8f15c73692c37dc3 P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:30.422963 7501 raft_consensus.cc:2454] T f485a2f4a2a6455e8f15c73692c37dc3 P bafdff0fd60146539631a476c4257e16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 051d464d644a4d4486462dfb18274c6a in term 1.
I20260807 19:42:30.422942 7351 raft_consensus.cc:2454] T f485a2f4a2a6455e8f15c73692c37dc3 P 35b5ecc2c9ef47c2bbe35653633e14da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 051d464d644a4d4486462dfb18274c6a in term 1.
I20260807 19:42:30.423460 7140 leader_election.cc:302] T f485a2f4a2a6455e8f15c73692c37dc3 P 051d464d644a4d4486462dfb18274c6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051d464d644a4d4486462dfb18274c6a, bafdff0fd60146539631a476c4257e16; no voters:
I20260807 19:42:30.423782 8320 raft_consensus.cc:2781] T f485a2f4a2a6455e8f15c73692c37dc3 P 051d464d644a4d4486462dfb18274c6a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:30.423947 8320 raft_consensus.cc:686] T f485a2f4a2a6455e8f15c73692c37dc3 P 051d464d644a4d4486462dfb18274c6a [term 1 LEADER]: Becoming Leader. State: Replica: 051d464d644a4d4486462dfb18274c6a, State: Running, Role: LEADER
I20260807 19:42:30.424278 8320 consensus_queue.cc:237] T f485a2f4a2a6455e8f15c73692c37dc3 P 051d464d644a4d4486462dfb18274c6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:42:30.863777 8627 consensus_queue.cc:1035] T 1b50e2d239144e2c80a169e79a235359 P 35b5ecc2c9ef47c2bbe35653633e14da [LEADER]: Connected to new peer: Peer: permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:30.871690 8630 consensus_queue.cc:1035] T 1b50e2d239144e2c80a169e79a235359 P 35b5ecc2c9ef47c2bbe35653633e14da [LEADER]: Connected to new peer: Peer: permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 19:42:30.885223 7287 consensus_peers.cc:487] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a45d0765a2c4d93ac9448496ce9c325. This is attempt 1: this message will repeat every 5th retry.
I20260807 19:42:30.893543 8320 consensus_queue.cc:1035] T 827e9a16eeb24a2cadf42fbb0b12a872 P 051d464d644a4d4486462dfb18274c6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:30.900202 8611 consensus_queue.cc:1035] T 827e9a16eeb24a2cadf42fbb0b12a872 P 051d464d644a4d4486462dfb18274c6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:30.937852 8611 consensus_queue.cc:1035] T fbea08c1f63a4a1c8a4d093be38f870e P 051d464d644a4d4486462dfb18274c6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:30.944226 8638 consensus_queue.cc:1035] T fbea08c1f63a4a1c8a4d093be38f870e P 051d464d644a4d4486462dfb18274c6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:31.018978 8642 raft_consensus.cc:482] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7b31edda0c4a48489ad60fc59c44ed31)
I20260807 19:42:31.019271 8642 raft_consensus.cc:504] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:31.020038 8642 leader_election.cc:288] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263), 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129)
W20260807 19:42:31.021308 8354 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111) [suppressed 9 similar messages]
W20260807 19:42:31.022744 8354 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:42:31.022943 8354 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129): Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: connect: Connection refused (error 111)
I20260807 19:42:31.023075 8354 leader_election.cc:302] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 959ac7807a8b42a59c61a23d4605c054; no voters: 1b782e30a8d44e7b94f080aa6327134f, 7b31edda0c4a48489ad60fc59c44ed31
I20260807 19:42:31.023402 8642 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260807 19:42:31.038898 6580 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskeMwJpl/build/asan/bin/kudu pid 8336
I20260807 19:42:31.047982 8399 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:31.078049 8628 consensus_queue.cc:1035] T 541fb835769b4d1f92407b288847cdee P bafdff0fd60146539631a476c4257e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:31.085206 8650 consensus_queue.cc:1035] T 541fb835769b4d1f92407b288847cdee P bafdff0fd60146539631a476c4257e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:31.174332 6580 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskeMwJpl/build/asan/bin/kudu with pid 8336
W20260807 19:42:31.198544 7139 connection.cc:535] client connection to 127.6.109.62:43417 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 19:42:31.318334 8638 consensus_queue.cc:1035] T f485a2f4a2a6455e8f15c73692c37dc3 P 051d464d644a4d4486462dfb18274c6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:31.324678 8611 consensus_queue.cc:1035] T f485a2f4a2a6455e8f15c73692c37dc3 P 051d464d644a4d4486462dfb18274c6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:31.815056 6580 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskeMwJpl/build/asan/bin/kudu
/tmp/dist-test-taskeMwJpl/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.109.62:43417
--webserver_interface=127.6.109.62
--webserver_port=37311
--builtin_ntp_servers=127.6.109.20:40195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129
--hive_metastore_uris=thrift://127.0.0.1:38987
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260807 19:42:32.186797 8662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:32.187141 8662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:32.187273 8662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:32.195029 8662 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:32.195163 8662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:32.195255 8662 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:32.195353 8662 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:32.205761 8662 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.109.20:40195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:38987
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.109.62:43417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.109.62
--webserver_port=37311
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:32.207288 8662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:32.209435 8662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:32.222468 8668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:32.222525 8667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:32.222430 8670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:32.223644 8662 server_base.cc:1034] running on GCE node
I20260807 19:42:32.224521 8662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:32.226176 8662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:32.227458 8662 hybrid_clock.cc:648] HybridClock initialized: now 1786131752227381 us; error 36 us; skew 500 ppm
I20260807 19:42:32.227849 8662 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:42:32.230541 8662 webserver.cc:460] Webserver started at http://127.6.109.62:37311/ using document root <none> and password file <none>
I20260807 19:42:32.231314 8662 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:32.231482 8662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:32.237941 8662 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:42:32.239302 8650 raft_consensus.cc:482] T d328d2e772ed4a6bb8597658f907f0f0 P bafdff0fd60146539631a476c4257e16 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 051d464d644a4d4486462dfb18274c6a)
I20260807 19:42:32.239544 8650 raft_consensus.cc:504] T d328d2e772ed4a6bb8597658f907f0f0 P bafdff0fd60146539631a476c4257e16 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:32.240258 8650 leader_election.cc:288] T d328d2e772ed4a6bb8597658f907f0f0 P bafdff0fd60146539631a476c4257e16 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467), 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483)
I20260807 19:42:32.241014 7351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d328d2e772ed4a6bb8597658f907f0f0" candidate_uuid: "bafdff0fd60146539631a476c4257e16" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" is_pre_election: true
W20260807 19:42:32.241503 7436 leader_election.cc:341] T d328d2e772ed4a6bb8597658f907f0f0 P bafdff0fd60146539631a476c4257e16 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Not found: Tablet not found: d328d2e772ed4a6bb8597658f907f0f0
I20260807 19:42:32.241852 7436 leader_election.cc:302] T d328d2e772ed4a6bb8597658f907f0f0 P bafdff0fd60146539631a476c4257e16 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bafdff0fd60146539631a476c4257e16; no voters: 051d464d644a4d4486462dfb18274c6a, 35b5ecc2c9ef47c2bbe35653633e14da
I20260807 19:42:32.241864 8676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:32.242172 8650 raft_consensus.cc:2726] T d328d2e772ed4a6bb8597658f907f0f0 P bafdff0fd60146539631a476c4257e16 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:42:32.243651 8662 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 19:42:32.243861 8662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/data,/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/wal
uuid: "959ac7807a8b42a59c61a23d4605c054"
format_stamp: "Formatted at 2026-08-07 19:42:04 on dist-test-slave-p3t5"
I20260807 19:42:32.245198 8662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:32.270478 8662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:32.271456 8662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:32.271770 8662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:32.290870 8662 rpc_server.cc:307] RPC server started. Bound to: 127.6.109.62:43417
I20260807 19:42:32.290890 8729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.109.62:43417 every 8 connection(s)
I20260807 19:42:32.292575 8662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-0/data/info.pb
I20260807 19:42:32.299795 8730 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:42:32.299960 6580 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskeMwJpl/build/asan/bin/kudu as pid 8662
I20260807 19:42:32.304967 8730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054: Bootstrap starting.
I20260807 19:42:32.324364 8730 log.cc:824] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:32.326532 7547 heartbeater.cc:347] Connected to a master server at 127.6.109.62:43417
I20260807 19:42:32.344341 7256 heartbeater.cc:347] Connected to a master server at 127.6.109.62:43417
I20260807 19:42:32.360993 8730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054: Bootstrap replayed 1/1 log segments. Stats: ops{read=40 overwritten=2 applied=38 ignored=0} inserts{seen=22 ignored=0} mutations{seen=25 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:42:32.361766 8730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054: Bootstrap complete.
I20260807 19:42:32.372876 8730 raft_consensus.cc:348] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 10 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:32.374389 7398 heartbeater.cc:347] Connected to a master server at 127.6.109.62:43417
I20260807 19:42:32.374362 8730 raft_consensus.cc:729] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 10 FOLLOWER]: Becoming Follower/Learner. State: Replica: 959ac7807a8b42a59c61a23d4605c054, State: Initialized, Role: FOLLOWER
I20260807 19:42:32.375025 8730 consensus_queue.cc:260] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 38, Last appended: 10.38, Last appended by leader: 38, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:32.376021 8738 sys_catalog.cc:455] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 10 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:32.376439 8738 sys_catalog.cc:458] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:32.377251 8730 sys_catalog.cc:564] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:32.397850 8752 catalog_manager.cc:1259] Loaded cluster ID: a80f7b000fe149f3a73e545a28c78fda
I20260807 19:42:32.397984 8752 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054: loading cluster ID for follower catalog manager: success
I20260807 19:42:32.405054 8752 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054: acquiring CA information for follower catalog manager: success
I20260807 19:42:32.408627 8752 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 19:42:32.400023 8662 master_runner.cc:186] Error getting master registration for 127.6.109.61:39263: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111),
I20260807 19:42:32.409557 8662 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:42:32.600020 8650 raft_consensus.cc:482] T 0d47a7a1285948468c0e6e1bddcd26d4 P bafdff0fd60146539631a476c4257e16 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 051d464d644a4d4486462dfb18274c6a)
I20260807 19:42:32.600314 8650 raft_consensus.cc:504] T 0d47a7a1285948468c0e6e1bddcd26d4 P bafdff0fd60146539631a476c4257e16 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:32.601130 8650 leader_election.cc:288] T 0d47a7a1285948468c0e6e1bddcd26d4 P bafdff0fd60146539631a476c4257e16 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467), 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483)
I20260807 19:42:32.601938 7351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d47a7a1285948468c0e6e1bddcd26d4" candidate_uuid: "bafdff0fd60146539631a476c4257e16" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" is_pre_election: true
W20260807 19:42:32.602341 7436 leader_election.cc:341] T 0d47a7a1285948468c0e6e1bddcd26d4 P bafdff0fd60146539631a476c4257e16 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Not found: Tablet not found: 0d47a7a1285948468c0e6e1bddcd26d4
I20260807 19:42:32.602634 7436 leader_election.cc:302] T 0d47a7a1285948468c0e6e1bddcd26d4 P bafdff0fd60146539631a476c4257e16 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bafdff0fd60146539631a476c4257e16; no voters: 051d464d644a4d4486462dfb18274c6a, 35b5ecc2c9ef47c2bbe35653633e14da
I20260807 19:42:32.603040 8650 raft_consensus.cc:2726] T 0d47a7a1285948468c0e6e1bddcd26d4 P bafdff0fd60146539631a476c4257e16 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:42:32.614760 8756 raft_consensus.cc:482] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 051d464d644a4d4486462dfb18274c6a)
I20260807 19:42:32.615031 8756 raft_consensus.cc:504] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:32.615789 8756 leader_election.cc:288] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
I20260807 19:42:32.616601 7501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d328d2e772ed4a6bb8597658f907f0f0" candidate_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bafdff0fd60146539631a476c4257e16" is_pre_election: true
W20260807 19:42:32.616901 7287 leader_election.cc:341] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Not found: Tablet not found: d328d2e772ed4a6bb8597658f907f0f0
I20260807 19:42:32.616948 7501 raft_consensus.cc:2454] T d328d2e772ed4a6bb8597658f907f0f0 P bafdff0fd60146539631a476c4257e16 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35b5ecc2c9ef47c2bbe35653633e14da in term 1.
I20260807 19:42:32.617443 7287 leader_election.cc:302] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 35b5ecc2c9ef47c2bbe35653633e14da, bafdff0fd60146539631a476c4257e16; no voters: 051d464d644a4d4486462dfb18274c6a
I20260807 19:42:32.617772 8756 raft_consensus.cc:2781] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:42:32.617900 8756 raft_consensus.cc:482] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da [term 1 FOLLOWER]: Starting leader election (detected failure of leader 051d464d644a4d4486462dfb18274c6a)
I20260807 19:42:32.618044 8756 raft_consensus.cc:3037] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:42:32.620078 8756 raft_consensus.cc:504] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:32.620816 8756 leader_election.cc:288] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 2 election: Requested vote from peers 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
I20260807 19:42:32.621526 7501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d328d2e772ed4a6bb8597658f907f0f0" candidate_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bafdff0fd60146539631a476c4257e16"
W20260807 19:42:32.621747 7287 leader_election.cc:341] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Not found: Tablet not found: d328d2e772ed4a6bb8597658f907f0f0
I20260807 19:42:32.621831 7501 raft_consensus.cc:3037] T d328d2e772ed4a6bb8597658f907f0f0 P bafdff0fd60146539631a476c4257e16 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:42:32.624923 7501 raft_consensus.cc:2454] T d328d2e772ed4a6bb8597658f907f0f0 P bafdff0fd60146539631a476c4257e16 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35b5ecc2c9ef47c2bbe35653633e14da in term 2.
I20260807 19:42:32.625478 7287 leader_election.cc:302] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 35b5ecc2c9ef47c2bbe35653633e14da, bafdff0fd60146539631a476c4257e16; no voters: 051d464d644a4d4486462dfb18274c6a
I20260807 19:42:32.625965 8756 raft_consensus.cc:2781] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:42:32.626416 8756 raft_consensus.cc:686] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da [term 2 LEADER]: Becoming Leader. State: Replica: 35b5ecc2c9ef47c2bbe35653633e14da, State: Running, Role: LEADER
I20260807 19:42:32.626816 8756 consensus_queue.cc:237] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:32.631547 8693 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" instance_seqno: 1786131727626128) as {username='slave'} at 127.6.109.2:54227; Asking this server to re-register.
I20260807 19:42:32.632441 7398 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:32.634439 8693 ts_manager.cc:194] Registered new tserver with Master: 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483)
I20260807 19:42:32.759588 8756 raft_consensus.cc:482] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 051d464d644a4d4486462dfb18274c6a)
I20260807 19:42:32.759871 8756 raft_consensus.cc:504] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:32.760628 8756 leader_election.cc:288] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
I20260807 19:42:32.761363 7501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d47a7a1285948468c0e6e1bddcd26d4" candidate_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bafdff0fd60146539631a476c4257e16" is_pre_election: true
W20260807 19:42:32.761715 7287 leader_election.cc:341] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Not found: Tablet not found: 0d47a7a1285948468c0e6e1bddcd26d4
I20260807 19:42:32.761902 7501 raft_consensus.cc:2454] T 0d47a7a1285948468c0e6e1bddcd26d4 P bafdff0fd60146539631a476c4257e16 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35b5ecc2c9ef47c2bbe35653633e14da in term 1.
I20260807 19:42:32.762441 7287 leader_election.cc:302] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 35b5ecc2c9ef47c2bbe35653633e14da, bafdff0fd60146539631a476c4257e16; no voters: 051d464d644a4d4486462dfb18274c6a
I20260807 19:42:32.762884 8756 raft_consensus.cc:2781] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:42:32.762991 8756 raft_consensus.cc:482] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da [term 1 FOLLOWER]: Starting leader election (detected failure of leader 051d464d644a4d4486462dfb18274c6a)
I20260807 19:42:32.763103 8756 raft_consensus.cc:3037] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:42:32.765867 8756 raft_consensus.cc:504] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:32.766520 8756 leader_election.cc:288] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 2 election: Requested vote from peers 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
I20260807 19:42:32.767190 7501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d47a7a1285948468c0e6e1bddcd26d4" candidate_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bafdff0fd60146539631a476c4257e16"
I20260807 19:42:32.767418 7501 raft_consensus.cc:3037] T 0d47a7a1285948468c0e6e1bddcd26d4 P bafdff0fd60146539631a476c4257e16 [term 1 FOLLOWER]: Advancing to term 2
W20260807 19:42:32.767402 7287 leader_election.cc:341] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Not found: Tablet not found: 0d47a7a1285948468c0e6e1bddcd26d4
I20260807 19:42:32.770298 7501 raft_consensus.cc:2454] T 0d47a7a1285948468c0e6e1bddcd26d4 P bafdff0fd60146539631a476c4257e16 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35b5ecc2c9ef47c2bbe35653633e14da in term 2.
I20260807 19:42:32.770763 7287 leader_election.cc:302] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 35b5ecc2c9ef47c2bbe35653633e14da, bafdff0fd60146539631a476c4257e16; no voters: 051d464d644a4d4486462dfb18274c6a
I20260807 19:42:32.771086 8756 raft_consensus.cc:2781] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:42:32.771281 8756 raft_consensus.cc:686] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da [term 2 LEADER]: Becoming Leader. State: Replica: 35b5ecc2c9ef47c2bbe35653633e14da, State: Running, Role: LEADER
I20260807 19:42:32.771584 8756 consensus_queue.cc:237] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } }
I20260807 19:42:32.806514 8738 raft_consensus.cc:482] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:32.806910 8738 raft_consensus.cc:504] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:32.809486 8738 leader_election.cc:288] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263), 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129)
W20260807 19:42:32.810276 8681 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:42:32.811952 8681 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:42:32.812271 8681 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129): Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: connect: Connection refused (error 111)
I20260807 19:42:32.812400 8681 leader_election.cc:302] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 959ac7807a8b42a59c61a23d4605c054; no voters: 1b782e30a8d44e7b94f080aa6327134f, 7b31edda0c4a48489ad60fc59c44ed31
I20260807 19:42:32.812795 8738 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260807 19:42:32.831975 8693 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "bafdff0fd60146539631a476c4257e16" instance_seqno: 1786131728431251) as {username='slave'} at 127.6.109.3:47533; Asking this server to re-register.
I20260807 19:42:32.832726 7547 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:32.834107 8693 ts_manager.cc:194] Registered new tserver with Master: bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
I20260807 19:42:32.848694 8693 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "051d464d644a4d4486462dfb18274c6a" instance_seqno: 1786131726838357) as {username='slave'} at 127.6.109.1:52981; Asking this server to re-register.
I20260807 19:42:32.849462 7256 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:32.851239 8693 ts_manager.cc:194] Registered new tserver with Master: 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467)
I20260807 19:42:33.506644 7501 raft_consensus.cc:1261] T d328d2e772ed4a6bb8597658f907f0f0 P bafdff0fd60146539631a476c4257e16 [term 2 FOLLOWER]: Refusing update from remote peer 35b5ecc2c9ef47c2bbe35653633e14da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:42:33.507352 8771 consensus_queue.cc:1035] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da [LEADER]: Connected to new peer: Peer: permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260807 19:42:33.513413 7287 consensus_peers.cc:487] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d328d2e772ed4a6bb8597658f907f0f0. This is attempt 1: this message will repeat every 5th retry.
I20260807 19:42:33.546856 8776 raft_consensus.cc:482] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:33.547093 8776 raft_consensus.cc:504] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:33.547871 8776 leader_election.cc:288] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263), 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129)
W20260807 19:42:33.550527 8681 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:42:33.550730 8681 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129): Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: connect: Connection refused (error 111)
I20260807 19:42:33.550822 8681 leader_election.cc:302] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 959ac7807a8b42a59c61a23d4605c054; no voters: 1b782e30a8d44e7b94f080aa6327134f, 7b31edda0c4a48489ad60fc59c44ed31
I20260807 19:42:33.551081 8776 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260807 19:42:33.672756 7501 raft_consensus.cc:1261] T 0d47a7a1285948468c0e6e1bddcd26d4 P bafdff0fd60146539631a476c4257e16 [term 2 FOLLOWER]: Refusing update from remote peer 35b5ecc2c9ef47c2bbe35653633e14da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:42:33.673447 8771 consensus_queue.cc:1035] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da [LEADER]: Connected to new peer: Peer: permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260807 19:42:33.678932 7287 consensus_peers.cc:487] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d47a7a1285948468c0e6e1bddcd26d4. This is attempt 1: this message will repeat every 5th retry.
I20260807 19:42:34.212435 8787 raft_consensus.cc:482] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:34.212714 8787 raft_consensus.cc:504] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:34.213513 8787 leader_election.cc:288] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263), 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129)
W20260807 19:42:34.216116 8681 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:42:34.216388 8681 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129): Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: connect: Connection refused (error 111)
I20260807 19:42:34.216506 8681 leader_election.cc:302] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 959ac7807a8b42a59c61a23d4605c054; no voters: 1b782e30a8d44e7b94f080aa6327134f, 7b31edda0c4a48489ad60fc59c44ed31
I20260807 19:42:34.216832 8787 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260807 19:42:34.821525 8798 raft_consensus.cc:482] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:34.821846 8798 raft_consensus.cc:504] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:34.822649 8798 leader_election.cc:288] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263), 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129)
W20260807 19:42:34.825407 8681 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:42:34.825685 8681 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129): Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: connect: Connection refused (error 111)
I20260807 19:42:34.825945 8681 leader_election.cc:302] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 959ac7807a8b42a59c61a23d4605c054; no voters: 1b782e30a8d44e7b94f080aa6327134f, 7b31edda0c4a48489ad60fc59c44ed31
I20260807 19:42:34.826278 8798 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260807 19:42:35.380910 6580 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskeMwJpl/build/asan/bin/kudu
/tmp/dist-test-taskeMwJpl/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.109.60:43129
--webserver_interface=127.6.109.60
--webserver_port=36383
--builtin_ntp_servers=127.6.109.20:40195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129
--hive_metastore_uris=thrift://127.0.0.1:38987
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260807 19:42:35.745024 8815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:35.745414 8815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:35.745559 8815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:35.753594 8815 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:35.753786 8815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:35.753890 8815 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:35.753999 8815 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:35.764597 8815 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.109.20:40195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:38987
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.109.60:43129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.109.60
--webserver_port=36383
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:35.766520 8815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:35.768417 8815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:35.782734 8824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:35.782658 8825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:35.782950 8828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:35.783429 8815 server_base.cc:1034] running on GCE node
I20260807 19:42:35.784379 8815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:35.786123 8815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:35.787397 8815 hybrid_clock.cc:648] HybridClock initialized: now 1786131755787330 us; error 46 us; skew 500 ppm
I20260807 19:42:35.787792 8815 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:42:35.790351 8815 webserver.cc:460] Webserver started at http://127.6.109.60:36383/ using document root <none> and password file <none>
I20260807 19:42:35.791139 8815 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:35.791303 8815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:35.797151 8815 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 19:42:35.800532 8834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:35.802168 8815 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 19:42:35.802402 8815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data,/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/wal
uuid: "7b31edda0c4a48489ad60fc59c44ed31"
format_stamp: "Formatted at 2026-08-07 19:42:05 on dist-test-slave-p3t5"
I20260807 19:42:35.803606 8815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:35.824931 8815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:35.826184 8815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:35.826524 8815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:35.847488 8815 rpc_server.cc:307] RPC server started. Bound to: 127.6.109.60:43129
I20260807 19:42:35.847514 8887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.109.60:43129 every 8 connection(s)
I20260807 19:42:35.849398 8815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data/info.pb
I20260807 19:42:35.855687 6580 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskeMwJpl/build/asan/bin/kudu as pid 8815
I20260807 19:42:35.860630 8889 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:42:35.866420 8889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: Bootstrap starting.
I20260807 19:42:35.871234 7255 heartbeater.cc:347] Connected to a master server at 127.6.109.60:43129
I20260807 19:42:35.880092 8889 log.cc:824] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:35.910598 8889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: Bootstrap replayed 1/1 log segments. Stats: ops{read=39 overwritten=1 applied=38 ignored=0} inserts{seen=22 ignored=0} mutations{seen=25 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:42:35.911258 8889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: Bootstrap complete.
I20260807 19:42:35.918938 8889 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 10 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:35.919916 8889 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 10 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b31edda0c4a48489ad60fc59c44ed31, State: Initialized, Role: FOLLOWER
I20260807 19:42:35.920550 8889 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 38, Last appended: 10.38, Last appended by leader: 38, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:35.921413 8894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 10 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:35.921898 8894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:35.922626 8889 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:35.936180 8909 catalog_manager.cc:1259] Loaded cluster ID: a80f7b000fe149f3a73e545a28c78fda
I20260807 19:42:35.936442 8909 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: loading cluster ID for follower catalog manager: success
I20260807 19:42:35.940815 8909 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: acquiring CA information for follower catalog manager: success
I20260807 19:42:35.943900 8909 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 19:42:35.965044 8815 master_runner.cc:186] Error getting master registration for 127.6.109.61:39263: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111),
I20260807 19:42:35.975732 8815 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:42:36.108901 7397 heartbeater.cc:347] Connected to a master server at 127.6.109.60:43129
I20260807 19:42:36.115031 8851 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" instance_seqno: 1786131727626128) as {username='slave'} at 127.6.109.2:40049; Asking this server to re-register.
I20260807 19:42:36.127476 7546 heartbeater.cc:347] Connected to a master server at 127.6.109.60:43129
I20260807 19:42:36.127703 7546 heartbeater.cc:503] Master 127.6.109.60:43129 was elected leader, sending a full tablet report...
I20260807 19:42:36.129988 7397 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:36.130065 8851 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "bafdff0fd60146539631a476c4257e16" instance_seqno: 1786131728431251) as {username='slave'} at 127.6.109.3:60783; Asking this server to re-register.
I20260807 19:42:36.132313 8851 ts_manager.cc:194] Registered new tserver with Master: 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483)
I20260807 19:42:36.135833 7546 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:36.137748 8851 ts_manager.cc:194] Registered new tserver with Master: bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
I20260807 19:42:36.376765 8851 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "051d464d644a4d4486462dfb18274c6a" instance_seqno: 1786131726838357) as {username='slave'} at 127.6.109.1:54275; Asking this server to re-register.
I20260807 19:42:36.377780 7255 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:36.379395 8851 ts_manager.cc:194] Registered new tserver with Master: 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467)
I20260807 19:42:36.520831 8920 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:36.521147 8920 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:36.523188 8920 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
I20260807 19:42:36.523977 8704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7b31edda0c4a48489ad60fc59c44ed31" candidate_term: 11 candidate_status { last_received { term: 10 index: 38 } } ignore_live_leader: false dest_uuid: "959ac7807a8b42a59c61a23d4605c054" is_pre_election: true
W20260807 19:42:36.524358 8838 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:42:36.524497 8704 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b31edda0c4a48489ad60fc59c44ed31 in term 10.
I20260807 19:42:36.525213 8837 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31, 959ac7807a8b42a59c61a23d4605c054; no voters:
I20260807 19:42:36.525671 8920 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260807 19:42:36.525883 8920 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:42:36.526032 8920 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 10 FOLLOWER]: Advancing to term 11
W20260807 19:42:36.526170 8838 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:42:36.530560 8920 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:36.531275 8920 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 11 election: Requested vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
I20260807 19:42:36.531810 8704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7b31edda0c4a48489ad60fc59c44ed31" candidate_term: 11 candidate_status { last_received { term: 10 index: 38 } } ignore_live_leader: false dest_uuid: "959ac7807a8b42a59c61a23d4605c054"
I20260807 19:42:36.532092 8704 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 10 FOLLOWER]: Advancing to term 11
W20260807 19:42:36.532814 8838 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:42:36.536412 8704 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b31edda0c4a48489ad60fc59c44ed31 in term 11.
I20260807 19:42:36.536953 8837 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31, 959ac7807a8b42a59c61a23d4605c054; no voters: 1b782e30a8d44e7b94f080aa6327134f
I20260807 19:42:36.537326 8920 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 11 FOLLOWER]: Leader election won for term 11
I20260807 19:42:36.537842 8920 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 11 LEADER]: Becoming Leader. State: Replica: 7b31edda0c4a48489ad60fc59c44ed31, State: Running, Role: LEADER
I20260807 19:42:36.538358 8920 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 38, Committed index: 38, Last appended: 10.38, Last appended by leader: 38, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:36.541390 8923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b31edda0c4a48489ad60fc59c44ed31. Latest consensus state: current_term: 11 leader_uuid: "7b31edda0c4a48489ad60fc59c44ed31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:36.541795 8923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:36.542541 8925 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:36.545161 8925 catalog_manager.cc:669] Loaded metadata for table default.table_472c6a33c87c4f1b9cdb926a2e5a6344 [id=0b2f82ef918b47d59b34a5b5757b9486]
I20260807 19:42:36.545720 8925 catalog_manager.cc:669] Loaded metadata for table default.table_c4966a160ae2456ebe5e58c41a9ac129 [id=490e9d5bbd5347aaae98c3a81cb3fe1e]
I20260807 19:42:36.546098 8925 catalog_manager.cc:669] Loaded metadata for table default.table_5d5e7f4976094d7a942e0ba75696c8f7 [id=9918b1d3805d424b92055cf267b8a4ab]
I20260807 19:42:36.546447 8925 catalog_manager.cc:669] Loaded metadata for table default.table_271147f124bf45a88d332b72ac8d3e13 [id=a75161def1a34954bf0846ab30a8cc38]
I20260807 19:42:36.546792 8925 catalog_manager.cc:669] Loaded metadata for table default.table_ca953b708a5a408898f72c483fc31ecf [id=bff436a0a60542b9b0fecacf4a6ed1b6]
I20260807 19:42:36.547150 8925 catalog_manager.cc:669] Loaded metadata for table default.table_49a9f7854b4c4847bce27e16296322b5 [id=cf4da31cb40447199dc079b8b6e709bc]
I20260807 19:42:36.549429 8925 tablet_loader.cc:96] loaded metadata for tablet 1b50e2d239144e2c80a169e79a235359 (table default.table_5d5e7f4976094d7a942e0ba75696c8f7 [id=9918b1d3805d424b92055cf267b8a4ab])
I20260807 19:42:36.549978 8925 tablet_loader.cc:96] loaded metadata for tablet 541fb835769b4d1f92407b288847cdee (table default.table_c4966a160ae2456ebe5e58c41a9ac129 [id=490e9d5bbd5347aaae98c3a81cb3fe1e])
I20260807 19:42:36.550499 8925 tablet_loader.cc:96] loaded metadata for tablet 807180c74470485182a926ab39d79ab4 (table default.table_ca953b708a5a408898f72c483fc31ecf [id=bff436a0a60542b9b0fecacf4a6ed1b6])
I20260807 19:42:36.550829 8925 tablet_loader.cc:96] loaded metadata for tablet 827e9a16eeb24a2cadf42fbb0b12a872 (table default.table_271147f124bf45a88d332b72ac8d3e13 [id=a75161def1a34954bf0846ab30a8cc38])
I20260807 19:42:36.551577 8925 tablet_loader.cc:96] loaded metadata for tablet f485a2f4a2a6455e8f15c73692c37dc3 (table default.table_472c6a33c87c4f1b9cdb926a2e5a6344 [id=0b2f82ef918b47d59b34a5b5757b9486])
I20260807 19:42:36.551877 8925 tablet_loader.cc:96] loaded metadata for tablet fbea08c1f63a4a1c8a4d093be38f870e (table default.table_49a9f7854b4c4847bce27e16296322b5 [id=cf4da31cb40447199dc079b8b6e709bc])
I20260807 19:42:36.552174 8925 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:36.553210 8925 catalog_manager.cc:1259] Loaded cluster ID: a80f7b000fe149f3a73e545a28c78fda
I20260807 19:42:36.553308 8925 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:36.554826 8925 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:36.555877 8925 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: Loaded TSK: 0
I20260807 19:42:36.556973 8925 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:42:36.558084 8925 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260807 19:42:36.559625 8925 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260807 19:42:36.627169 8851 master_service.cc:946] ReplaceTablet: received request to replace tablet 807180c74470485182a926ab39d79ab4 from {username='slave'} at 127.0.0.1:60032
I20260807 19:42:36.627216 8852 master_service.cc:946] ReplaceTablet: received request to replace tablet 248c1670db7244cfac99dd4817b513a4 from {username='slave'} at 127.0.0.1:60032
I20260807 19:42:36.627646 8847 master_service.cc:946] ReplaceTablet: received request to replace tablet 807180c74470485182a926ab39d79ab4 from {username='slave'} at 127.0.0.1:60032
I20260807 19:42:36.628047 8850 master_service.cc:946] ReplaceTablet: received request to replace tablet d328d2e772ed4a6bb8597658f907f0f0 from {username='slave'} at 127.0.0.1:60032
I20260807 19:42:36.629078 8850 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60032:
table { table_name: "default.Table_5d5e7f4976094d7a942e0ba75696c8f7" } modify_external_catalogs: true
I20260807 19:42:36.630151 8850 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60032:
table { table_name: "default.Table_5d5e7f4976094d7a942e0ba75696c8f7" } modify_external_catalogs: true
I20260807 19:42:36.629894 8848 master_service.cc:946] ReplaceTablet: received request to replace tablet 9a45d0765a2c4d93ac9448496ce9c325 from {username='slave'} at 127.0.0.1:60032
I20260807 19:42:36.629609 8845 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60032:
table { table_name: "default.Table_271147f124bf45a88d332b72ac8d3e13" } new_table_name: "default.Table_830c2e1165884867b9cb12c80f090b48" modify_external_catalogs: true
I20260807 19:42:36.631110 8846 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60032:
table { table_name: "default.Table_49a9f7854b4c4847bce27e16296322b5" } new_table_name: "default.Table_e60fcc04a04e4a0d9456e55b94757d74" modify_external_catalogs: true
I20260807 19:42:36.630765 8852 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60032:
name: "default.Table_8ea588612f6c47c79ac5c31aa592809e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:36.633893 8843 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60032:
name: "default.Table_61514c4f10b54148a57bbaf295205ace"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:36.629729 8844 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60032:
table { table_name: "default.Table_c4966a160ae2456ebe5e58c41a9ac129" } new_table_name: "default.Table_d6db13f17872439a8858b3d5fd56bfde" modify_external_catalogs: true
I20260807 19:42:36.634058 8842 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60032:
name: "default.Table_1455c4f504ff4603bc11da85f16cc69f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:36.634166 8848 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60032:
name: "default.Table_24957fcd29dc4a3db1b0ab2dcf131779"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260807 19:42:36.639824 8908 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_c4966a160ae2456ebe5e58c41a9ac129
*** Check failure stack trace: ***
@ 0x7f4e1be3e9ad google::LogMessage::Fail() at ??:0
I20260807 19:42:36.643814 8704 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 11 FOLLOWER]: Refusing update from remote peer 7b31edda0c4a48489ad60fc59c44ed31: Log matching property violated. Preceding OpId in replica: term: 10 index: 38. Preceding OpId from leader: term: 11 index: 40. (index mismatch)
@ 0x7f4e1be4188a google::LogMessage::SendToLog() at ??:0
I20260807 19:42:36.645229 8920 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 39, Last known committed idx: 38, Time since last communication: 0.000s
W20260807 19:42:36.645670 8838 consensus_peers.cc:487] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 -> Peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Couldn't send request to peer 1b782e30a8d44e7b94f080aa6327134f. Status: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
@ 0x7f4e1be3e4de google::LogMessage::Flush() at ??:0
@ 0x7f4e1be3e6d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7f4e319dcba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7f4e319db901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f4e319e266c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260807 19:42:36.669055 8929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b31edda0c4a48489ad60fc59c44ed31. Latest consensus state: current_term: 11 leader_uuid: "7b31edda0c4a48489ad60fc59c44ed31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:36.669523 8929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:36.672189 8930 mvcc.cc:205] Tried to move back new op lower bound from 7315995675191042048 to 7315995674784931840. Current Snapshot: MvccSnapshot[applied={T|T < 7315995675191042048}]
I20260807 19:42:36.672461 8923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 11 leader_uuid: "7b31edda0c4a48489ad60fc59c44ed31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:36.672744 8923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: This master's current role is: LEADER
@ 0x7f4e319e24e9 std::_Function_handler<>::_M_invoke() at ??:0
I20260807 19:42:36.674566 8851 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: Sending DeleteTablet for 3 replicas of tablet 807180c74470485182a926ab39d79ab4
@ 0x9edb7d std::function<>::operator()() at ??:?
I20260807 19:42:36.677243 8851 catalog_manager.cc:6565] ReplaceTablet: tablet 807180c74470485182a926ab39d79ab4 deleted and replaced by tablet 9811a05ede794d0c9dd7ab6bfa089a6f
I20260807 19:42:36.678231 8847 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60032:
table { table_name: "default.Table_ca953b708a5a408898f72c483fc31ecf" } new_table_name: "default.Table_eb8c970c75a34df9a37b1812e7f5cc76" modify_external_catalogs: true
I20260807 19:42:36.680953 8929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 11 leader_uuid: "7b31edda0c4a48489ad60fc59c44ed31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:36.681334 8929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:36.690023 8851 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60032:
name: "default.Table_8461316ca98b4afab4331f54eb8d5600"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
@ 0x7f4e1ea68eaf kudu::Thread::SuperviseThread() at ??:0
I20260807 19:42:36.699898 7328 tablet_service.cc:1514] Processing DeleteTablet for tablet 807180c74470485182a926ab39d79ab4 with delete_type TABLET_DATA_DELETED (replaced by tablet 9811a05ede794d0c9dd7ab6bfa089a6f) from {username='slave'} at 127.0.0.1:56674
I20260807 19:42:36.700385 7330 tablet_service.cc:1467] Processing CreateTablet for tablet 9811a05ede794d0c9dd7ab6bfa089a6f (DEFAULT_TABLE table=default.table_ca953b708a5a408898f72c483fc31ecf [id=bff436a0a60542b9b0fecacf4a6ed1b6]), partition=RANGE (key) PARTITION UNBOUNDED
@ 0x7f4e21d426db start_thread at ??:0
I20260807 19:42:36.700877 7330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9811a05ede794d0c9dd7ab6bfa089a6f. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7f4e1acd861f clone at ??:0
I20260807 19:42:36.703958 8943 tablet_replica.cc:330] T 807180c74470485182a926ab39d79ab4 P 35b5ecc2c9ef47c2bbe35653633e14da: stopping tablet replica
I20260807 19:42:36.704236 8943 raft_consensus.cc:2229] T 807180c74470485182a926ab39d79ab4 P 35b5ecc2c9ef47c2bbe35653633e14da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:36.704525 8943 raft_consensus.cc:2258] T 807180c74470485182a926ab39d79ab4 P 35b5ecc2c9ef47c2bbe35653633e14da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:36.705157 8943 ts_tablet_manager.cc:1905] T 807180c74470485182a926ab39d79ab4 P 35b5ecc2c9ef47c2bbe35653633e14da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:36.714257 8943 ts_tablet_manager.cc:1918] T 807180c74470485182a926ab39d79ab4 P 35b5ecc2c9ef47c2bbe35653633e14da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260807 19:42:36.714504 8943 log.cc:1196] T 807180c74470485182a926ab39d79ab4 P 35b5ecc2c9ef47c2bbe35653633e14da: Deleting WAL directory at /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/ts-1/wal/wals/807180c74470485182a926ab39d79ab4
I20260807 19:42:36.715374 8943 ts_tablet_manager.cc:1939] T 807180c74470485182a926ab39d79ab4 P 35b5ecc2c9ef47c2bbe35653633e14da: Deleting consensus metadata
I20260807 19:42:36.717896 8944 tablet_bootstrap.cc:492] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da: Bootstrap starting.
I20260807 19:42:36.721127 8944 tablet_bootstrap.cc:654] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:36.724694 8944 tablet_bootstrap.cc:492] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da: No bootstrap required, opened a new log
I20260807 19:42:36.724907 8944 ts_tablet_manager.cc:1397] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.006s
I20260807 19:42:36.725878 8944 raft_consensus.cc:348] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:42:36.726189 8944 raft_consensus.cc:374] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:36.726339 8944 raft_consensus.cc:729] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35b5ecc2c9ef47c2bbe35653633e14da, State: Initialized, Role: FOLLOWER
I20260807 19:42:36.726634 8944 consensus_queue.cc:260] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:42:36.729234 8944 ts_tablet_manager.cc:1428] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
W20260807 19:42:36.946556 8940 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:42:36.680462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47492 (local address 127.6.109.1:44467)
0807 19:42:36.686322 (+ 5860us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:36.686332 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:36.697021 (+ 10689us) server_negotiation.cc:407] Connection header received
0807 19:42:36.697162 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:36.697167 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:36.697303 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:36.697413 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:36.703798 (+ 6385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:36.704765 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:36.946225 (+241460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47492: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":5635,"thread_start_us":5427,"threads_started":1}
W20260807 19:42:36.946955 8938 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:42:36.678877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51642 (local address 127.6.109.3:40225)
0807 19:42:36.690735 (+ 11858us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:36.690744 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:36.690776 (+ 32us) server_negotiation.cc:407] Connection header received
0807 19:42:36.690853 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:36.690858 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:36.690948 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:36.691067 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:36.699718 (+ 8651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:36.701087 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:36.946427 (+245340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51642: BlockingRecv error: recv got EOF from 127.0.0.1:51642 (error 108)
Metrics: {"server-negotiator.queue_time_us":11686,"thread_start_us":11542,"threads_started":1}
W20260807 19:42:36.947190 7436 connection.cc:535] client connection to 127.6.109.60:43129 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 19:42:36.947456 7559 connection.cc:535] client connection to 127.6.109.60:43129 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 19:42:36.947688 7286 connection.cc:535] server connection from 127.0.0.1:56674 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 19:42:36.948084 7287 connection.cc:535] client connection to 127.6.109.60:43129 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 19:42:36.948102 7140 connection.cc:535] client connection to 127.6.109.60:43129 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 19:42:36.948314 7559 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.6.109.60:43129)
W20260807 19:42:36.949445 7559 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.6.109.60:43129)
W20260807 19:42:36.950199 7559 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.6.109.60:43129)
W20260807 19:42:36.951288 7559 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.6.109.60:43129)
I20260807 19:42:37.168601 8813 raft_consensus.cc:482] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:37.168845 8813 raft_consensus.cc:504] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:42:37.169608 8813 leader_election.cc:288] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
W20260807 19:42:37.170925 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
W20260807 19:42:37.171188 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
I20260807 19:42:37.171307 7287 leader_election.cc:302] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35b5ecc2c9ef47c2bbe35653633e14da; no voters: 051d464d644a4d4486462dfb18274c6a, bafdff0fd60146539631a476c4257e16
I20260807 19:42:37.171597 8813 raft_consensus.cc:2726] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:42:37.192111 8960 raft_consensus.cc:482] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7b31edda0c4a48489ad60fc59c44ed31)
I20260807 19:42:37.192399 8960 raft_consensus.cc:504] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:37.193077 8960 leader_election.cc:288] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263), 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129)
W20260807 19:42:37.195819 8681 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:42:37.196054 8681 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129): Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: connect: Connection refused (error 111)
I20260807 19:42:37.196141 8681 leader_election.cc:302] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 959ac7807a8b42a59c61a23d4605c054; no voters: 1b782e30a8d44e7b94f080aa6327134f, 7b31edda0c4a48489ad60fc59c44ed31
I20260807 19:42:37.196384 8960 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260807 19:42:37.330090 7140 consensus_peers.cc:487] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a -> Peer 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483): Couldn't send request to peer 35b5ecc2c9ef47c2bbe35653633e14da. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 807180c74470485182a926ab39d79ab4. This is attempt 1: this message will repeat every 5th retry.
I20260807 19:42:37.412766 8662 master_runner.cc:186] Error getting master registration for 127.6.109.61:39263: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111),
I20260807 19:42:37.412976 8662 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:42:37.811149 8973 raft_consensus.cc:482] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7b31edda0c4a48489ad60fc59c44ed31)
I20260807 19:42:37.811410 8973 raft_consensus.cc:504] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:37.812120 8973 leader_election.cc:288] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263), 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129)
W20260807 19:42:37.813051 8681 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111) [suppressed 9 similar messages]
W20260807 19:42:37.814597 8681 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:42:37.814831 8681 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129): Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: connect: Connection refused (error 111)
I20260807 19:42:37.814960 8681 leader_election.cc:302] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 959ac7807a8b42a59c61a23d4605c054; no voters: 1b782e30a8d44e7b94f080aa6327134f, 7b31edda0c4a48489ad60fc59c44ed31
I20260807 19:42:37.815296 8973 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260807 19:42:38.526494 8987 raft_consensus.cc:482] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7b31edda0c4a48489ad60fc59c44ed31)
I20260807 19:42:38.526784 8987 raft_consensus.cc:504] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:38.527521 8987 leader_election.cc:288] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263), 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129)
W20260807 19:42:38.530222 8681 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:42:38.530530 8681 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129): Network error: Client connection negotiation failed: client connection to 127.6.109.60:43129: connect: Connection refused (error 111)
I20260807 19:42:38.530683 8681 leader_election.cc:302] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 959ac7807a8b42a59c61a23d4605c054; no voters: 1b782e30a8d44e7b94f080aa6327134f, 7b31edda0c4a48489ad60fc59c44ed31
I20260807 19:42:38.530936 8987 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260807 19:42:39.345814 6580 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskeMwJpl/build/asan/bin/kudu
/tmp/dist-test-taskeMwJpl/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.109.60:43129
--webserver_interface=127.6.109.60
--webserver_port=36383
--builtin_ntp_servers=127.6.109.20:40195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129
--hive_metastore_uris=thrift://127.0.0.1:38987
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260807 19:42:39.712852 9005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:39.713212 9005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:39.713297 9005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:39.721293 9005 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:39.721427 9005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:39.721472 9005 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:39.721522 9005 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:39.731421 9005 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.109.20:40195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:38987
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.109.60:43129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.109.60
--webserver_port=36383
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:39.732848 9005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:39.734797 9005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:39.747668 9015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:39.747761 9013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:39.747759 9017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:39.754395 9005 server_base.cc:1034] running on GCE node
I20260807 19:42:39.755395 9005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:39.757144 9005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:39.758436 9005 hybrid_clock.cc:648] HybridClock initialized: now 1786131759758349 us; error 54 us; skew 500 ppm
I20260807 19:42:39.758924 9005 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:42:39.766544 9005 webserver.cc:460] Webserver started at http://127.6.109.60:36383/ using document root <none> and password file <none>
I20260807 19:42:39.767455 9005 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:39.767647 9005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:39.773983 9005 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:42:39.777339 9023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:39.779103 9005 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:42:39.779362 9005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data,/tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/wal
uuid: "7b31edda0c4a48489ad60fc59c44ed31"
format_stamp: "Formatted at 2026-08-07 19:42:05 on dist-test-slave-p3t5"
I20260807 19:42:39.780522 9005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:39.798312 9005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:39.799345 9005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:39.799669 9005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:39.820235 9005 rpc_server.cc:307] RPC server started. Bound to: 127.6.109.60:43129
I20260807 19:42:39.820291 9075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.109.60:43129 every 8 connection(s)
I20260807 19:42:39.821925 9005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskeMwJpl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1786131708164198-6580-0/minicluster-data/master-2/data/info.pb
I20260807 19:42:39.829464 9076 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:42:39.831513 6580 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskeMwJpl/build/asan/bin/kudu as pid 9005
I20260807 19:42:39.835719 9076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: Bootstrap starting.
I20260807 19:42:39.850683 9076 log.cc:824] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:39.859167 7546 heartbeater.cc:347] Connected to a master server at 127.6.109.60:43129
I20260807 19:42:39.888276 9076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: Bootstrap replayed 1/1 log segments. Stats: ops{read=42 overwritten=1 applied=41 ignored=0} inserts{seen=23 ignored=0} mutations{seen=27 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:42:39.888890 9076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: Bootstrap complete.
I20260807 19:42:39.898656 9076 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 11 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:39.899704 9076 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 11 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b31edda0c4a48489ad60fc59c44ed31, State: Initialized, Role: FOLLOWER
I20260807 19:42:39.900426 9076 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 41, Last appended: 11.41, Last appended by leader: 41, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:39.901304 9082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 11 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:39.901708 9082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:39.902681 9076 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:39.917797 9098 catalog_manager.cc:1259] Loaded cluster ID: a80f7b000fe149f3a73e545a28c78fda
I20260807 19:42:39.918053 9098 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: loading cluster ID for follower catalog manager: success
I20260807 19:42:39.923517 9098 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: acquiring CA information for follower catalog manager: success
I20260807 19:42:39.927213 9098 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 19:42:39.942119 9005 master_runner.cc:186] Error getting master registration for 127.6.109.61:39263: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111),
I20260807 19:42:39.951581 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:42:39.966353 7255 heartbeater.cc:347] Connected to a master server at 127.6.109.60:43129
I20260807 19:42:39.969491 9039 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "051d464d644a4d4486462dfb18274c6a" instance_seqno: 1786131726838357) as {username='slave'} at 127.6.109.1:48895; Asking this server to re-register.
I20260807 19:42:39.970834 7255 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:39.973131 9039 ts_manager.cc:194] Registered new tserver with Master: 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467)
I20260807 19:42:40.038196 7397 heartbeater.cc:347] Connected to a master server at 127.6.109.60:43129
I20260807 19:42:40.040174 9039 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" instance_seqno: 1786131727626128) as {username='slave'} at 127.6.109.2:34277; Asking this server to re-register.
I20260807 19:42:40.040868 7397 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:40.042371 9039 ts_manager.cc:194] Registered new tserver with Master: 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483)
I20260807 19:42:40.060319 9106 raft_consensus.cc:482] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7b31edda0c4a48489ad60fc59c44ed31)
I20260807 19:42:40.060539 9106 raft_consensus.cc:504] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:40.061483 9106 leader_election.cc:288] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263), 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129)
W20260807 19:42:40.064852 8681 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:42:40.070386 9051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "959ac7807a8b42a59c61a23d4605c054" candidate_term: 12 candidate_status { last_received { term: 11 index: 41 } } ignore_live_leader: false dest_uuid: "7b31edda0c4a48489ad60fc59c44ed31" is_pre_election: true
I20260807 19:42:40.071110 9051 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 959ac7807a8b42a59c61a23d4605c054 in term 11.
I20260807 19:42:40.071776 8681 leader_election.cc:302] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31, 959ac7807a8b42a59c61a23d4605c054; no voters: 1b782e30a8d44e7b94f080aa6327134f
I20260807 19:42:40.072165 9106 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260807 19:42:40.072314 9106 raft_consensus.cc:482] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7b31edda0c4a48489ad60fc59c44ed31)
I20260807 19:42:40.072445 9106 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 11 FOLLOWER]: Advancing to term 12
I20260807 19:42:40.074668 9106 raft_consensus.cc:504] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:40.075421 9106 leader_election.cc:288] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 12 election: Requested vote from peers 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263), 7b31edda0c4a48489ad60fc59c44ed31 (127.6.109.60:43129)
I20260807 19:42:40.076256 9051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "959ac7807a8b42a59c61a23d4605c054" candidate_term: 12 candidate_status { last_received { term: 11 index: 41 } } ignore_live_leader: false dest_uuid: "7b31edda0c4a48489ad60fc59c44ed31"
I20260807 19:42:40.076538 9051 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 11 FOLLOWER]: Advancing to term 12
W20260807 19:42:40.078047 8681 leader_election.cc:334] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:42:40.084333 9051 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 959ac7807a8b42a59c61a23d4605c054 in term 12.
I20260807 19:42:40.085014 8681 leader_election.cc:302] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31, 959ac7807a8b42a59c61a23d4605c054; no voters: 1b782e30a8d44e7b94f080aa6327134f
I20260807 19:42:40.085470 9106 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 12 FOLLOWER]: Leader election won for term 12
I20260807 19:42:40.086093 9106 raft_consensus.cc:686] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [term 12 LEADER]: Becoming Leader. State: Replica: 959ac7807a8b42a59c61a23d4605c054, State: Running, Role: LEADER
I20260807 19:42:40.086664 9106 consensus_queue.cc:237] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 41, Committed index: 41, Last appended: 11.41, Last appended by leader: 41, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:40.088678 9110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 959ac7807a8b42a59c61a23d4605c054. Latest consensus state: current_term: 12 leader_uuid: "959ac7807a8b42a59c61a23d4605c054" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:40.089144 9110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:40.090085 9112 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:40.092947 9112 catalog_manager.cc:669] Loaded metadata for table default.table_472c6a33c87c4f1b9cdb926a2e5a6344 [id=0b2f82ef918b47d59b34a5b5757b9486]
I20260807 19:42:40.093468 9112 catalog_manager.cc:669] Loaded metadata for table default.table_c4966a160ae2456ebe5e58c41a9ac129 [id=490e9d5bbd5347aaae98c3a81cb3fe1e]
I20260807 19:42:40.093918 9112 catalog_manager.cc:669] Loaded metadata for table default.table_5d5e7f4976094d7a942e0ba75696c8f7 [id=9918b1d3805d424b92055cf267b8a4ab]
I20260807 19:42:40.094326 9112 catalog_manager.cc:669] Loaded metadata for table default.table_271147f124bf45a88d332b72ac8d3e13 [id=a75161def1a34954bf0846ab30a8cc38]
I20260807 19:42:40.094691 9112 catalog_manager.cc:669] Loaded metadata for table default.table_ca953b708a5a408898f72c483fc31ecf [id=bff436a0a60542b9b0fecacf4a6ed1b6]
I20260807 19:42:40.095098 9112 catalog_manager.cc:669] Loaded metadata for table default.table_49a9f7854b4c4847bce27e16296322b5 [id=cf4da31cb40447199dc079b8b6e709bc]
I20260807 19:42:40.097391 9112 tablet_loader.cc:96] loaded metadata for tablet 1b50e2d239144e2c80a169e79a235359 (table default.table_5d5e7f4976094d7a942e0ba75696c8f7 [id=9918b1d3805d424b92055cf267b8a4ab])
I20260807 19:42:40.098029 9112 tablet_loader.cc:96] loaded metadata for tablet 541fb835769b4d1f92407b288847cdee (table default.table_c4966a160ae2456ebe5e58c41a9ac129 [id=490e9d5bbd5347aaae98c3a81cb3fe1e])
I20260807 19:42:40.098686 9112 tablet_loader.cc:96] loaded metadata for tablet 827e9a16eeb24a2cadf42fbb0b12a872 (table default.table_271147f124bf45a88d332b72ac8d3e13 [id=a75161def1a34954bf0846ab30a8cc38])
I20260807 19:42:40.099022 9112 tablet_loader.cc:96] loaded metadata for tablet 9811a05ede794d0c9dd7ab6bfa089a6f (table default.table_ca953b708a5a408898f72c483fc31ecf [id=bff436a0a60542b9b0fecacf4a6ed1b6])
I20260807 19:42:40.099766 9112 tablet_loader.cc:96] loaded metadata for tablet f485a2f4a2a6455e8f15c73692c37dc3 (table default.table_472c6a33c87c4f1b9cdb926a2e5a6344 [id=0b2f82ef918b47d59b34a5b5757b9486])
I20260807 19:42:40.100099 9112 tablet_loader.cc:96] loaded metadata for tablet fbea08c1f63a4a1c8a4d093be38f870e (table default.table_49a9f7854b4c4847bce27e16296322b5 [id=cf4da31cb40447199dc079b8b6e709bc])
I20260807 19:42:40.100400 9112 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:40.101555 9112 catalog_manager.cc:1259] Loaded cluster ID: a80f7b000fe149f3a73e545a28c78fda
I20260807 19:42:40.101688 9112 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:40.103660 9112 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:40.104992 9112 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054: Loaded TSK: 0
I20260807 19:42:40.106225 9112 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:42:40.107316 9112 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260807 19:42:40.108297 9112 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260807 19:42:40.172505 8693 master_service.cc:946] ReplaceTablet: received request to replace tablet 827e9a16eeb24a2cadf42fbb0b12a872 from {username='slave'} at 127.0.0.1:45986
I20260807 19:42:40.172955 8691 master_service.cc:946] ReplaceTablet: received request to replace tablet 827e9a16eeb24a2cadf42fbb0b12a872 from {username='slave'} at 127.0.0.1:45986
I20260807 19:42:40.174314 8685 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45986:
name: "default.Table_f2d1ca595d0c43ec982c1377f39155ee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:40.175093 8694 master_service.cc:946] ReplaceTablet: received request to replace tablet 827e9a16eeb24a2cadf42fbb0b12a872 from {username='slave'} at 127.0.0.1:45986
I20260807 19:42:40.175458 8692 master_service.cc:946] ReplaceTablet: received request to replace tablet 248c1670db7244cfac99dd4817b513a4 from {username='slave'} at 127.0.0.1:45986
I20260807 19:42:40.176589 8692 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45986:
table { table_name: "default.Table_49a9f7854b4c4847bce27e16296322b5" } new_table_name: "default.Table_e60fcc04a04e4a0d9456e55b94757d74" modify_external_catalogs: true
I20260807 19:42:40.177500 8690 master_service.cc:946] ReplaceTablet: received request to replace tablet 0d47a7a1285948468c0e6e1bddcd26d4 from {username='slave'} at 127.0.0.1:45986
I20260807 19:42:40.178256 8690 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45986:
table { table_name: "default.Table_271147f124bf45a88d332b72ac8d3e13" } new_table_name: "default.Table_830c2e1165884867b9cb12c80f090b48" modify_external_catalogs: true
I20260807 19:42:40.179121 8689 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45986:
name: "default.Table_b6eca8c0167f4d80b6ef4f8125aea68f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260807 19:42:40.182451 8751 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_c4966a160ae2456ebe5e58c41a9ac129
*** Check failure stack trace: ***
I20260807 19:42:40.182926 8688 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45986:
name: "default.Table_8cc3d0e85b19426f8df95f1894914926"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:40.183828 8687 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45986:
name: "default.Table_5d300b1b9ec343b88bbb7481707c963a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:40.184710 8686 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45986:
table { table_name: "default.Table_472c6a33c87c4f1b9cdb926a2e5a6344" } new_table_name: "default.Table_cda45f853cca4f52a3371bbaec9c7750" modify_external_catalogs: true
@ 0x7f1a9d30a9ad google::LogMessage::Fail() at ??:0
@ 0x7f1a9d30d88a google::LogMessage::SendToLog() at ??:0
@ 0x7f1a9d30a4de google::LogMessage::Flush() at ??:0
@ 0x7f1a9d30a6d1 google::LogMessage::~LogMessage() at ??:0
I20260807 19:42:40.197839 9051 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Refusing update from remote peer 959ac7807a8b42a59c61a23d4605c054: Log matching property violated. Preceding OpId in replica: term: 11 index: 41. Preceding OpId from leader: term: 12 index: 43. (index mismatch)
@ 0x7f1ab2ea8ba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260807 19:42:40.198944 9110 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 42, Last known committed idx: 41, Time since last communication: 0.000s
@ 0x7f1ab2ea7901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
W20260807 19:42:40.207784 8681 consensus_peers.cc:487] T 00000000000000000000000000000000 P 959ac7807a8b42a59c61a23d4605c054 -> Peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Couldn't send request to peer 1b782e30a8d44e7b94f080aa6327134f. Status: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
@ 0x7f1ab2eae66c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7f1ab2eae4e9 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f1a9ff34eaf kudu::Thread::SuperviseThread() at ??:0
I20260807 19:42:40.226719 9082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 959ac7807a8b42a59c61a23d4605c054. Latest consensus state: current_term: 12 leader_uuid: "959ac7807a8b42a59c61a23d4605c054" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:40.227135 9082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:40.230302 9117 mvcc.cc:205] Tried to move back new op lower bound from 7315995689758396416 to 7315995689319206912. Current Snapshot: MvccSnapshot[applied={T|T < 7315995689758396416}]
@ 0x7f1aa320e6db start_thread at ??:0
@ 0x7f1a9c1a461f clone at ??:0
I20260807 19:42:40.237744 9082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 12 leader_uuid: "959ac7807a8b42a59c61a23d4605c054" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } } }
I20260807 19:42:40.238070 9082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:40.344102 8991 raft_consensus.cc:482] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:40.344352 8991 raft_consensus.cc:504] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:42:40.345206 8991 leader_election.cc:288] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
W20260807 19:42:40.346431 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
W20260807 19:42:40.346670 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
I20260807 19:42:40.346815 7287 leader_election.cc:302] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35b5ecc2c9ef47c2bbe35653633e14da; no voters: 051d464d644a4d4486462dfb18274c6a, bafdff0fd60146539631a476c4257e16
I20260807 19:42:40.347383 8992 raft_consensus.cc:2726] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:42:40.364753 9039 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "bafdff0fd60146539631a476c4257e16" instance_seqno: 1786131728431251) as {username='slave'} at 127.6.109.3:34779; Asking this server to re-register.
I20260807 19:42:40.365834 7546 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:40.368212 9039 ts_manager.cc:194] Registered new tserver with Master: bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
W20260807 19:42:40.458894 9027 connection.cc:535] server connection from 127.0.0.1:60744 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 19:42:40.459481 7435 connection.cc:535] client connection to 127.6.109.62:43417 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 19:42:40.459504 7558 connection.cc:535] client connection to 127.6.109.62:43417 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 19:42:40.460228 7558 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.6.109.62:43417)
W20260807 19:42:40.460250 7139 connection.cc:535] client connection to 127.6.109.62:43417 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 19:42:40.460594 7286 connection.cc:535] client connection to 127.6.109.62:43417 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 19:42:40.460721 7558 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.6.109.62:43417)
W20260807 19:42:40.461102 7558 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.6.109.62:43417)
W20260807 19:42:40.463110 7558 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.6.109.62:43417)
W20260807 19:42:40.521744 7436 consensus_peers.cc:487] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 248c1670db7244cfac99dd4817b513a4. This is attempt 6: this message will repeat every 5th retry.
I20260807 19:42:40.729903 9082 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:42:40.730188 9082 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:40.732383 9082 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:42:40.733457 9026 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
W20260807 19:42:40.735183 9026 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
W20260807 19:42:40.735410 9027 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:42:40.735635 9027 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:42:40.736058 9082 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260807 19:42:41.168186 7287 consensus_peers.cc:487] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a45d0765a2c4d93ac9448496ce9c325. This is attempt 6: this message will repeat every 5th retry.
I20260807 19:42:41.309912 9145 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:42:41.310151 9145 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:41.310985 9145 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:42:41.313179 9026 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
W20260807 19:42:41.313441 9027 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:42:41.313661 9027 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:42:41.313961 9145 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260807 19:42:41.898713 9156 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:42:41.898932 9156 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:41.899574 9156 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:42:41.901894 9027 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:42:41.902119 9026 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
I20260807 19:42:41.902310 9026 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:42:41.902611 9156 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260807 19:42:42.071938 6580 master-stress-test.cc:578] Killing test thread 1/25
W20260807 19:42:42.866441 7287 consensus_peers.cc:487] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d328d2e772ed4a6bb8597658f907f0f0. This is attempt 6: this message will repeat every 5th retry.
I20260807 19:42:43.227607 9175 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:42:43.227902 9175 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:43.228681 9175 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:42:43.231096 9026 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
W20260807 19:42:43.231312 9027 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:42:43.231426 9027 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:42:43.231729 9175 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260807 19:42:43.394320 7287 consensus_peers.cc:487] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d47a7a1285948468c0e6e1bddcd26d4. This is attempt 6: this message will repeat every 5th retry.
I20260807 19:42:44.511806 9199 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:42:44.512080 9199 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:44.512923 9199 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:42:44.515565 9026 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
W20260807 19:42:44.515798 9027 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:42:44.515970 9027 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:42:44.516283 9199 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260807 19:42:44.953169 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:42:44.953423 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:42:45.728513 9212 raft_consensus.cc:482] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:45.728955 9212 raft_consensus.cc:504] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:42:45.730638 9212 leader_election.cc:288] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
W20260807 19:42:45.732509 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
W20260807 19:42:45.732957 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
I20260807 19:42:45.733254 7287 leader_election.cc:302] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35b5ecc2c9ef47c2bbe35653633e14da; no voters: 051d464d644a4d4486462dfb18274c6a, bafdff0fd60146539631a476c4257e16
I20260807 19:42:45.733870 9212 raft_consensus.cc:2726] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260807 19:42:47.417464 7140 consensus_peers.cc:487] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a -> Peer 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483): Couldn't send request to peer 35b5ecc2c9ef47c2bbe35653633e14da. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 807180c74470485182a926ab39d79ab4. This is attempt 6: this message will repeat every 5th retry.
I20260807 19:42:48.179883 9255 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:42:48.180168 9255 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:48.180989 9255 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:42:48.182076 9026 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111) [suppressed 9 similar messages]
W20260807 19:42:48.183478 9027 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:42:48.183694 9026 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
I20260807 19:42:48.183863 9026 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:42:48.184168 9255 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260807 19:42:49.955163 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:42:49.955405 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 19:42:50.757359 7436 consensus_peers.cc:487] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 248c1670db7244cfac99dd4817b513a4. This is attempt 11: this message will repeat every 5th retry.
W20260807 19:42:50.927537 7287 consensus_peers.cc:487] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a45d0765a2c4d93ac9448496ce9c325. This is attempt 11: this message will repeat every 5th retry.
I20260807 19:42:51.858285 9296 raft_consensus.cc:482] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:51.858544 9296 raft_consensus.cc:504] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:42:51.859321 9296 leader_election.cc:288] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
W20260807 19:42:51.860602 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
W20260807 19:42:51.860869 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
I20260807 19:42:51.861028 7287 leader_election.cc:302] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35b5ecc2c9ef47c2bbe35653633e14da; no voters: 051d464d644a4d4486462dfb18274c6a, bafdff0fd60146539631a476c4257e16
I20260807 19:42:51.861363 9296 raft_consensus.cc:2726] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:42:51.921267 9309 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:42:51.921533 9309 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:51.922324 9309 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:42:51.925052 9027 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:42:51.925319 9026 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
I20260807 19:42:51.925470 9026 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:42:51.925773 9309 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260807 19:42:52.371316 7287 consensus_peers.cc:487] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d328d2e772ed4a6bb8597658f907f0f0. This is attempt 11: this message will repeat every 5th retry.
W20260807 19:42:53.681891 7287 consensus_peers.cc:487] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d47a7a1285948468c0e6e1bddcd26d4. This is attempt 11: this message will repeat every 5th retry.
I20260807 19:42:54.957044 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:42:54.957226 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:42:56.803953 9352 raft_consensus.cc:482] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:56.804239 9352 raft_consensus.cc:504] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:42:56.805002 9352 leader_election.cc:288] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
W20260807 19:42:56.806120 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
W20260807 19:42:56.806397 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
I20260807 19:42:56.806488 7287 leader_election.cc:302] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35b5ecc2c9ef47c2bbe35653633e14da; no voters: 051d464d644a4d4486462dfb18274c6a, bafdff0fd60146539631a476c4257e16
I20260807 19:42:56.806859 9352 raft_consensus.cc:2726] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260807 19:42:56.870836 7140 consensus_peers.cc:487] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a -> Peer 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483): Couldn't send request to peer 35b5ecc2c9ef47c2bbe35653633e14da. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 807180c74470485182a926ab39d79ab4. This is attempt 11: this message will repeat every 5th retry.
I20260807 19:42:57.724874 9394 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:42:57.725190 9394 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:57.726035 9394 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:42:57.727134 9026 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260807 19:42:57.728876 9027 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:42:57.729100 9026 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
I20260807 19:42:57.729274 9026 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:42:57.729557 9394 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260807 19:42:59.081434 9415 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:42:59.081751 9415 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:42:59.082500 9415 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:42:59.084977 9026 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
W20260807 19:42:59.085166 9027 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:42:59.085354 9027 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:42:59.085664 9415 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260807 19:42:59.958878 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:42:59.959095 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 19:43:00.370213 7287 consensus_peers.cc:487] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a45d0765a2c4d93ac9448496ce9c325. This is attempt 16: this message will repeat every 5th retry.
W20260807 19:43:01.501816 7436 consensus_peers.cc:487] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 248c1670db7244cfac99dd4817b513a4. This is attempt 16: this message will repeat every 5th retry.
W20260807 19:43:02.221082 7287 consensus_peers.cc:487] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d328d2e772ed4a6bb8597658f907f0f0. This is attempt 16: this message will repeat every 5th retry.
W20260807 19:43:04.139416 7287 consensus_peers.cc:487] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d47a7a1285948468c0e6e1bddcd26d4. This is attempt 16: this message will repeat every 5th retry.
I20260807 19:43:04.862228 9513 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:43:04.862432 9513 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:43:04.863068 9513 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:43:04.864209 9027 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260807 19:43:04.865880 9026 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
W20260807 19:43:04.866098 9027 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:43:04.866214 9027 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:43:04.866472 9513 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260807 19:43:04.960458 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:43:04.960647 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 19:43:07.010298 7140 consensus_peers.cc:487] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a -> Peer 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483): Couldn't send request to peer 35b5ecc2c9ef47c2bbe35653633e14da. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 807180c74470485182a926ab39d79ab4. This is attempt 16: this message will repeat every 5th retry.
W20260807 19:43:08.928954 7398 heartbeater.cc:644] Failed to heartbeat to 127.6.109.62:43417 (108 consecutive failures): Network error: Failed to ping master at 127.6.109.62:43417: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
W20260807 19:43:09.212194 7547 heartbeater.cc:644] Failed to heartbeat to 127.6.109.62:43417 (72 consecutive failures): Network error: Failed to ping master at 127.6.109.62:43417: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
W20260807 19:43:09.302465 7256 heartbeater.cc:644] Failed to heartbeat to 127.6.109.62:43417 (70 consecutive failures): Network error: Failed to ping master at 127.6.109.62:43417: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
I20260807 19:43:09.962556 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:43:09.962755 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:43:10.271793 9604 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:43:10.272023 9604 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:43:10.272769 9604 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:43:10.273820 9027 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260807 19:43:10.275334 9026 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
W20260807 19:43:10.275527 9027 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:43:10.275657 9027 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:43:10.275880 9604 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260807 19:43:10.494197 7287 consensus_peers.cc:487] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a45d0765a2c4d93ac9448496ce9c325. This is attempt 21: this message will repeat every 5th retry.
W20260807 19:43:11.584141 7436 consensus_peers.cc:487] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 248c1670db7244cfac99dd4817b513a4. This is attempt 21: this message will repeat every 5th retry.
W20260807 19:43:11.731065 7287 consensus_peers.cc:487] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d328d2e772ed4a6bb8597658f907f0f0. This is attempt 21: this message will repeat every 5th retry.
W20260807 19:43:14.475821 7287 consensus_peers.cc:487] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d47a7a1285948468c0e6e1bddcd26d4. This is attempt 21: this message will repeat every 5th retry.
I20260807 19:43:14.964573 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:43:14.964775 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:43:16.582413 9659 raft_consensus.cc:482] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:43:16.582738 9659 raft_consensus.cc:504] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:43:16.583890 9659 leader_election.cc:288] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
W20260807 19:43:16.584879 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
W20260807 19:43:16.585343 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
I20260807 19:43:16.585563 7287 leader_election.cc:302] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35b5ecc2c9ef47c2bbe35653633e14da; no voters: 051d464d644a4d4486462dfb18274c6a, bafdff0fd60146539631a476c4257e16
I20260807 19:43:16.585997 9659 raft_consensus.cc:2726] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260807 19:43:16.586143 7140 consensus_peers.cc:487] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a -> Peer 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483): Couldn't send request to peer 35b5ecc2c9ef47c2bbe35653633e14da. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 807180c74470485182a926ab39d79ab4. This is attempt 21: this message will repeat every 5th retry.
I20260807 19:43:19.966437 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:43:19.966655 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 19:43:20.526054 7287 consensus_peers.cc:487] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a45d0765a2c4d93ac9448496ce9c325. This is attempt 26: this message will repeat every 5th retry.
I20260807 19:43:20.585840 9774 raft_consensus.cc:482] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:43:20.586104 9774 raft_consensus.cc:504] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:43:20.586834 9774 leader_election.cc:288] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
W20260807 19:43:20.587821 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
W20260807 19:43:20.588052 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
I20260807 19:43:20.588145 7287 leader_election.cc:302] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35b5ecc2c9ef47c2bbe35653633e14da; no voters: 051d464d644a4d4486462dfb18274c6a, bafdff0fd60146539631a476c4257e16
I20260807 19:43:20.588395 9774 raft_consensus.cc:2726] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:43:21.685804 9798 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:43:21.686062 9798 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:43:21.686919 9798 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:43:21.688254 9026 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260807 19:43:21.689599 9027 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:43:21.689843 9026 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
I20260807 19:43:21.690018 9026 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:43:21.690346 9798 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260807 19:43:21.859022 7436 consensus_peers.cc:487] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 248c1670db7244cfac99dd4817b513a4. This is attempt 26: this message will repeat every 5th retry.
W20260807 19:43:21.896683 7287 consensus_peers.cc:487] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d328d2e772ed4a6bb8597658f907f0f0. This is attempt 26: this message will repeat every 5th retry.
I20260807 19:43:22.836549 9810 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:43:22.836750 9810 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:43:22.837523 9810 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:43:22.839742 9026 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
W20260807 19:43:22.839991 9027 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:43:22.840138 9027 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:43:22.840418 9810 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260807 19:43:24.167189 7287 consensus_peers.cc:487] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d47a7a1285948468c0e6e1bddcd26d4. This is attempt 26: this message will repeat every 5th retry.
I20260807 19:43:24.968207 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:43:24.968405 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:43:26.686277 9870 raft_consensus.cc:482] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:43:26.686559 9870 raft_consensus.cc:504] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:43:26.687260 9870 leader_election.cc:288] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
W20260807 19:43:26.688485 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
W20260807 19:43:26.688799 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
I20260807 19:43:26.688992 7287 leader_election.cc:302] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35b5ecc2c9ef47c2bbe35653633e14da; no voters: 051d464d644a4d4486462dfb18274c6a, bafdff0fd60146539631a476c4257e16
I20260807 19:43:26.689360 9870 raft_consensus.cc:2726] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260807 19:43:27.216675 7140 consensus_peers.cc:487] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a -> Peer 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483): Couldn't send request to peer 35b5ecc2c9ef47c2bbe35653633e14da. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 807180c74470485182a926ab39d79ab4. This is attempt 26: this message will repeat every 5th retry.
I20260807 19:43:29.516374 9905 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:43:29.516681 9905 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:43:29.517853 9905 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:43:29.520012 9026 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260807 19:43:29.521867 9027 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:43:29.522107 9026 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
I20260807 19:43:29.522348 9026 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:43:29.522787 9905 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260807 19:43:29.970067 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:43:29.970373 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 19:43:30.525897 7287 consensus_peers.cc:487] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a45d0765a2c4d93ac9448496ce9c325. This is attempt 31: this message will repeat every 5th retry.
W20260807 19:43:32.246515 7436 consensus_peers.cc:487] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 248c1670db7244cfac99dd4817b513a4. This is attempt 31: this message will repeat every 5th retry.
W20260807 19:43:32.331768 7287 consensus_peers.cc:487] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d328d2e772ed4a6bb8597658f907f0f0. This is attempt 31: this message will repeat every 5th retry.
W20260807 19:43:33.176913 7287 consensus_peers.cc:487] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d47a7a1285948468c0e6e1bddcd26d4. This is attempt 31: this message will repeat every 5th retry.
I20260807 19:43:34.972134 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:43:34.972332 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 19:43:36.944662 7140 consensus_peers.cc:487] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a -> Peer 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483): Couldn't send request to peer 35b5ecc2c9ef47c2bbe35653633e14da. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 807180c74470485182a926ab39d79ab4. This is attempt 31: this message will repeat every 5th retry.
I20260807 19:43:39.975301 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:43:39.975628 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 19:43:40.554553 7287 consensus_peers.cc:487] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a45d0765a2c4d93ac9448496ce9c325. This is attempt 36: this message will repeat every 5th retry.
I20260807 19:43:41.297080 10084 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:43:41.297317 10084 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:43:41.298168 10084 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:43:41.299501 9026 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260807 19:43:41.301319 9026 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
W20260807 19:43:41.301523 9027 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:43:41.301829 9027 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:43:41.302173 10084 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260807 19:43:42.166235 7287 consensus_peers.cc:487] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d328d2e772ed4a6bb8597658f907f0f0. This is attempt 36: this message will repeat every 5th retry.
W20260807 19:43:42.563874 7287 consensus_peers.cc:487] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d47a7a1285948468c0e6e1bddcd26d4. This is attempt 36: this message will repeat every 5th retry.
I20260807 19:43:42.611850 10106 raft_consensus.cc:482] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:43:42.612151 10106 raft_consensus.cc:504] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:43:42.613029 10106 leader_election.cc:288] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
W20260807 19:43:42.614250 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
W20260807 19:43:42.614518 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
I20260807 19:43:42.614652 7287 leader_election.cc:302] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35b5ecc2c9ef47c2bbe35653633e14da; no voters: 051d464d644a4d4486462dfb18274c6a, bafdff0fd60146539631a476c4257e16
I20260807 19:43:42.615002 10106 raft_consensus.cc:2726] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260807 19:43:43.041867 7436 consensus_peers.cc:487] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 248c1670db7244cfac99dd4817b513a4. This is attempt 36: this message will repeat every 5th retry.
I20260807 19:43:44.977422 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:43:44.977735 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 19:43:46.203611 7140 consensus_peers.cc:487] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a -> Peer 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483): Couldn't send request to peer 35b5ecc2c9ef47c2bbe35653633e14da. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 807180c74470485182a926ab39d79ab4. This is attempt 36: this message will repeat every 5th retry.
I20260807 19:43:49.639725 10235 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:43:49.639997 10235 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:43:49.641019 10235 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:43:49.643011 9026 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260807 19:43:49.644622 9026 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
W20260807 19:43:49.644865 9027 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:43:49.645092 9027 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:43:49.645445 10235 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260807 19:43:49.979413 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:43:49.979660 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 19:43:50.662974 7287 consensus_peers.cc:487] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a45d0765a2c4d93ac9448496ce9c325. This is attempt 41: this message will repeat every 5th retry.
W20260807 19:43:51.782555 7287 consensus_peers.cc:487] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d328d2e772ed4a6bb8597658f907f0f0. This is attempt 41: this message will repeat every 5th retry.
W20260807 19:43:53.100725 7287 consensus_peers.cc:487] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d47a7a1285948468c0e6e1bddcd26d4. This is attempt 41: this message will repeat every 5th retry.
W20260807 19:43:53.281216 7436 consensus_peers.cc:487] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 248c1670db7244cfac99dd4817b513a4. This is attempt 41: this message will repeat every 5th retry.
I20260807 19:43:54.468923 10313 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:43:54.469178 10313 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:43:54.470139 10313 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:43:54.472772 9026 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
W20260807 19:43:54.473067 9027 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:43:54.473282 9027 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:43:54.473583 10313 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260807 19:43:54.981204 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:43:54.981426 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 19:43:56.600966 7140 consensus_peers.cc:487] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a -> Peer 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483): Couldn't send request to peer 35b5ecc2c9ef47c2bbe35653633e14da. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 807180c74470485182a926ab39d79ab4. This is attempt 41: this message will repeat every 5th retry.
I20260807 19:43:59.983773 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:43:59.984017 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 19:44:01.068497 7287 consensus_peers.cc:487] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a45d0765a2c4d93ac9448496ce9c325. This is attempt 46: this message will repeat every 5th retry.
I20260807 19:44:01.341956 10410 raft_consensus.cc:482] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:44:01.342259 10410 raft_consensus.cc:504] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:44:01.343662 10410 leader_election.cc:288] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
W20260807 19:44:01.344897 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
W20260807 19:44:01.345238 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
I20260807 19:44:01.345474 7287 leader_election.cc:302] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35b5ecc2c9ef47c2bbe35653633e14da; no voters: 051d464d644a4d4486462dfb18274c6a, bafdff0fd60146539631a476c4257e16
I20260807 19:44:01.345834 10410 raft_consensus.cc:2726] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260807 19:44:02.156581 7287 consensus_peers.cc:487] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d328d2e772ed4a6bb8597658f907f0f0. This is attempt 46: this message will repeat every 5th retry.
W20260807 19:44:02.945433 7287 consensus_peers.cc:487] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d47a7a1285948468c0e6e1bddcd26d4. This is attempt 46: this message will repeat every 5th retry.
W20260807 19:44:03.171069 7436 consensus_peers.cc:487] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 248c1670db7244cfac99dd4817b513a4. This is attempt 46: this message will repeat every 5th retry.
I20260807 19:44:04.992863 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:44:04.993155 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 19:44:06.469831 7140 consensus_peers.cc:487] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a -> Peer 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483): Couldn't send request to peer 35b5ecc2c9ef47c2bbe35653633e14da. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 807180c74470485182a926ab39d79ab4. This is attempt 46: this message will repeat every 5th retry.
W20260807 19:44:09.164582 7398 heartbeater.cc:644] Failed to heartbeat to 127.6.109.62:43417 (341 consecutive failures): Network error: Failed to ping master at 127.6.109.62:43417: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
I20260807 19:44:09.539827 10545 raft_consensus.cc:482] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:44:09.540131 10545 raft_consensus.cc:504] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:44:09.541097 10545 leader_election.cc:288] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
W20260807 19:44:09.542414 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
W20260807 19:44:09.542765 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
I20260807 19:44:09.543040 7287 leader_election.cc:302] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35b5ecc2c9ef47c2bbe35653633e14da; no voters: 051d464d644a4d4486462dfb18274c6a, bafdff0fd60146539631a476c4257e16
I20260807 19:44:09.543417 10545 raft_consensus.cc:2726] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260807 19:44:09.591782 7262 heartbeater.cc:644] Failed to heartbeat to 127.6.109.61:39263 (286 consecutive failures): Network error: Failed to ping master at 127.6.109.61:39263: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:44:09.693421 7547 heartbeater.cc:644] Failed to heartbeat to 127.6.109.62:43417 (226 consecutive failures): Network error: Failed to ping master at 127.6.109.62:43417: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
I20260807 19:44:09.995460 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:44:09.995697 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 19:44:11.278514 7287 consensus_peers.cc:487] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a45d0765a2c4d93ac9448496ce9c325. This is attempt 51: this message will repeat every 5th retry.
W20260807 19:44:12.818308 7287 consensus_peers.cc:487] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d328d2e772ed4a6bb8597658f907f0f0. This is attempt 51: this message will repeat every 5th retry.
W20260807 19:44:13.108093 7436 consensus_peers.cc:487] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 248c1670db7244cfac99dd4817b513a4. This is attempt 51: this message will repeat every 5th retry.
I20260807 19:44:13.807051 10617 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:44:13.807508 10617 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:44:13.808775 10617 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:44:13.810336 9027 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260807 19:44:13.812079 9026 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
W20260807 19:44:13.812387 9027 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:44:13.812575 9027 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:44:13.812958 10617 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260807 19:44:13.905041 7287 consensus_peers.cc:487] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d47a7a1285948468c0e6e1bddcd26d4. This is attempt 51: this message will repeat every 5th retry.
I20260807 19:44:14.997968 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:44:14.998236 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:44:15.068434 10639 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:44:15.068797 10639 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:44:15.069836 10639 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:44:15.072167 9026 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
W20260807 19:44:15.072434 9027 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:44:15.072609 9027 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:44:15.072974 10639 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260807 19:44:17.204131 7140 consensus_peers.cc:487] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a -> Peer 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483): Couldn't send request to peer 35b5ecc2c9ef47c2bbe35653633e14da. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 807180c74470485182a926ab39d79ab4. This is attempt 51: this message will repeat every 5th retry.
I20260807 19:44:20.000316 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:44:20.000579 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:44:21.562263 10728 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:44:21.562526 10728 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:44:21.563331 10728 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:44:21.566895 9027 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260807 19:44:21.568630 9026 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
W20260807 19:44:21.568892 9027 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:44:21.569133 9027 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:44:21.569501 10728 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260807 19:44:21.773134 7287 consensus_peers.cc:487] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a45d0765a2c4d93ac9448496ce9c325. This is attempt 56: this message will repeat every 5th retry.
I20260807 19:44:22.157111 10703 raft_consensus.cc:482] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:44:22.157333 10703 raft_consensus.cc:504] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:44:22.158471 10703 leader_election.cc:288] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
W20260807 19:44:22.159703 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
W20260807 19:44:22.160148 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
I20260807 19:44:22.160317 7287 leader_election.cc:302] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35b5ecc2c9ef47c2bbe35653633e14da; no voters: 051d464d644a4d4486462dfb18274c6a, bafdff0fd60146539631a476c4257e16
I20260807 19:44:22.160755 10703 raft_consensus.cc:2726] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260807 19:44:22.707800 7287 consensus_peers.cc:487] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d328d2e772ed4a6bb8597658f907f0f0. This is attempt 56: this message will repeat every 5th retry.
W20260807 19:44:22.883517 7436 consensus_peers.cc:487] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 248c1670db7244cfac99dd4817b513a4. This is attempt 56: this message will repeat every 5th retry.
W20260807 19:44:23.500768 7287 consensus_peers.cc:487] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d47a7a1285948468c0e6e1bddcd26d4. This is attempt 56: this message will repeat every 5th retry.
I20260807 19:44:25.002521 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:44:25.002759 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:44:25.146668 10792 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:44:25.147014 10792 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:44:25.147867 10792 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:44:25.151625 9027 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:44:25.151872 9026 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
I20260807 19:44:25.152154 9026 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:44:25.152536 10792 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260807 19:44:26.865907 7140 consensus_peers.cc:487] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a -> Peer 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483): Couldn't send request to peer 35b5ecc2c9ef47c2bbe35653633e14da. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 807180c74470485182a926ab39d79ab4. This is attempt 56: this message will repeat every 5th retry.
I20260807 19:44:30.005214 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:44:30.005455 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:44:31.210614 10878 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:44:31.210975 10878 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:44:31.211890 10878 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:44:31.213704 9027 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260807 19:44:31.215372 9026 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
W20260807 19:44:31.215536 9027 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:44:31.215706 9027 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:44:31.216030 10878 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260807 19:44:31.622843 7287 consensus_peers.cc:487] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a45d0765a2c4d93ac9448496ce9c325. This is attempt 61: this message will repeat every 5th retry.
W20260807 19:44:32.266525 7287 consensus_peers.cc:487] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d328d2e772ed4a6bb8597658f907f0f0. This is attempt 61: this message will repeat every 5th retry.
W20260807 19:44:33.795827 7436 consensus_peers.cc:487] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 248c1670db7244cfac99dd4817b513a4. This is attempt 61: this message will repeat every 5th retry.
W20260807 19:44:34.457487 7287 consensus_peers.cc:487] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d47a7a1285948468c0e6e1bddcd26d4. This is attempt 61: this message will repeat every 5th retry.
I20260807 19:44:35.007177 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:44:35.007426 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 19:44:37.023581 7140 consensus_peers.cc:487] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a -> Peer 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483): Couldn't send request to peer 35b5ecc2c9ef47c2bbe35653633e14da. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 807180c74470485182a926ab39d79ab4. This is attempt 61: this message will repeat every 5th retry.
I20260807 19:44:40.002293 9029 maintenance_manager.cc:419] P 7b31edda0c4a48489ad60fc59c44ed31: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033361
I20260807 19:44:40.009454 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:44:40.009833 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:44:40.066690 9028 maintenance_manager.cc:643] P 7b31edda0c4a48489ad60fc59c44ed31: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.063s user 0.027s sys 0.016s Metrics: {"bytes_written":12389,"cfile_init":1,"dirs.queue_time_us":761,"dirs.run_cpu_time_us":275,"dirs.run_wall_time_us":1171,"drs_written":1,"lbm_read_time_us":73,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1195,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":24,"thread_start_us":1136,"threads_started":2,"wal-append.queue_time_us":767}
I20260807 19:44:40.068491 9029 maintenance_manager.cc:419] P 7b31edda0c4a48489ad60fc59c44ed31: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1360 bytes on disk
I20260807 19:44:40.069473 9028 maintenance_manager.cc:643] P 7b31edda0c4a48489ad60fc59c44ed31: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":80,"lbm_reads_lt_1ms":4}
W20260807 19:44:40.723299 7287 consensus_peers.cc:487] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a45d0765a2c4d93ac9448496ce9c325. This is attempt 66: this message will repeat every 5th retry.
I20260807 19:44:41.843263 11048 raft_consensus.cc:482] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:44:41.843568 11048 raft_consensus.cc:504] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:44:41.844777 11048 leader_election.cc:288] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
W20260807 19:44:41.845963 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
W20260807 19:44:41.846431 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
I20260807 19:44:41.846602 7287 leader_election.cc:302] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35b5ecc2c9ef47c2bbe35653633e14da; no voters: 051d464d644a4d4486462dfb18274c6a, bafdff0fd60146539631a476c4257e16
I20260807 19:44:41.847002 11048 raft_consensus.cc:2726] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260807 19:44:42.997009 7287 consensus_peers.cc:487] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d328d2e772ed4a6bb8597658f907f0f0. This is attempt 66: this message will repeat every 5th retry.
W20260807 19:44:43.702105 7436 consensus_peers.cc:487] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 248c1670db7244cfac99dd4817b513a4. This is attempt 66: this message will repeat every 5th retry.
W20260807 19:44:44.623068 7287 consensus_peers.cc:487] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d47a7a1285948468c0e6e1bddcd26d4. This is attempt 66: this message will repeat every 5th retry.
I20260807 19:44:45.011644 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:44:45.011868 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 19:44:47.134696 7140 consensus_peers.cc:487] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a -> Peer 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483): Couldn't send request to peer 35b5ecc2c9ef47c2bbe35653633e14da. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 807180c74470485182a926ab39d79ab4. This is attempt 66: this message will repeat every 5th retry.
I20260807 19:44:48.278527 11160 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:44:48.279001 11160 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:44:48.280133 11160 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:44:48.282442 9026 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260807 19:44:48.284564 9026 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
W20260807 19:44:48.284966 9027 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:44:48.285204 9027 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:44:48.285652 11160 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260807 19:44:49.101518 11175 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:44:49.101835 11175 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:44:49.102651 11175 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:44:49.106026 9026 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
W20260807 19:44:49.106302 9027 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:44:49.106539 9027 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:44:49.106952 11175 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260807 19:44:50.013957 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:44:50.014287 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 19:44:50.853869 7287 consensus_peers.cc:487] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a45d0765a2c4d93ac9448496ce9c325. This is attempt 71: this message will repeat every 5th retry.
I20260807 19:44:52.344936 11241 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:44:52.345352 11241 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:44:52.346144 11241 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:44:52.348935 9027 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:44:52.349121 9026 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
I20260807 19:44:52.349360 9026 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:44:52.349737 11241 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260807 19:44:53.195840 7287 consensus_peers.cc:487] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d328d2e772ed4a6bb8597658f907f0f0. This is attempt 71: this message will repeat every 5th retry.
W20260807 19:44:53.756662 7436 consensus_peers.cc:487] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 248c1670db7244cfac99dd4817b513a4. This is attempt 71: this message will repeat every 5th retry.
W20260807 19:44:54.088693 7287 consensus_peers.cc:487] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d47a7a1285948468c0e6e1bddcd26d4. This is attempt 71: this message will repeat every 5th retry.
I20260807 19:44:55.016239 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:44:55.016549 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:44:56.245596 11279 raft_consensus.cc:482] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:44:56.245893 11279 raft_consensus.cc:504] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:44:56.246846 11279 leader_election.cc:288] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
W20260807 19:44:56.248180 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
W20260807 19:44:56.248466 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
I20260807 19:44:56.248626 7287 leader_election.cc:302] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35b5ecc2c9ef47c2bbe35653633e14da; no voters: 051d464d644a4d4486462dfb18274c6a, bafdff0fd60146539631a476c4257e16
I20260807 19:44:56.248992 11279 raft_consensus.cc:2726] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260807 19:44:57.884397 7140 consensus_peers.cc:487] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a -> Peer 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483): Couldn't send request to peer 35b5ecc2c9ef47c2bbe35653633e14da. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 807180c74470485182a926ab39d79ab4. This is attempt 71: this message will repeat every 5th retry.
I20260807 19:45:00.019798 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:45:00.020104 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:45:00.052865 11383 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:45:00.053246 11383 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:45:00.054468 11383 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:45:00.055222 9026 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111) [suppressed 5 similar messages]
W20260807 19:45:00.057829 9027 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:45:00.058115 9026 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
I20260807 19:45:00.058333 9026 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:45:00.058651 11383 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260807 19:45:00.555334 7287 consensus_peers.cc:487] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a45d0765a2c4d93ac9448496ce9c325. This is attempt 76: this message will repeat every 5th retry.
W20260807 19:45:03.132506 7287 consensus_peers.cc:487] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d328d2e772ed4a6bb8597658f907f0f0. This is attempt 76: this message will repeat every 5th retry.
W20260807 19:45:03.532418 7436 consensus_peers.cc:487] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 248c1670db7244cfac99dd4817b513a4. This is attempt 76: this message will repeat every 5th retry.
W20260807 19:45:04.385378 7287 consensus_peers.cc:487] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d47a7a1285948468c0e6e1bddcd26d4. This is attempt 76: this message will repeat every 5th retry.
I20260807 19:45:05.022363 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:45:05.022675 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 19:45:07.492055 7140 consensus_peers.cc:487] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a -> Peer 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483): Couldn't send request to peer 35b5ecc2c9ef47c2bbe35653633e14da. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 807180c74470485182a926ab39d79ab4. This is attempt 76: this message will repeat every 5th retry.
W20260807 19:45:09.306612 7399 heartbeater.cc:644] Failed to heartbeat to 127.6.109.61:39263 (641 consecutive failures): Network error: Failed to ping master at 127.6.109.61:39263: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:45:09.853462 7262 heartbeater.cc:644] Failed to heartbeat to 127.6.109.61:39263 (438 consecutive failures): Network error: Failed to ping master at 127.6.109.61:39263: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:45:09.892485 7548 heartbeater.cc:644] Failed to heartbeat to 127.6.109.61:39263 (447 consecutive failures): Network error: Failed to ping master at 127.6.109.61:39263: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:45:10.024597 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:45:10.024825 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 19:45:10.529091 7287 consensus_peers.cc:487] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a45d0765a2c4d93ac9448496ce9c325. This is attempt 81: this message will repeat every 5th retry.
W20260807 19:45:12.935724 7436 consensus_peers.cc:487] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 248c1670db7244cfac99dd4817b513a4. This is attempt 81: this message will repeat every 5th retry.
I20260807 19:45:12.952020 11647 raft_consensus.cc:482] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:45:12.952426 11647 raft_consensus.cc:504] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:45:12.953493 11647 leader_election.cc:288] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
W20260807 19:45:12.954766 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
W20260807 19:45:12.955077 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
I20260807 19:45:12.955216 7287 leader_election.cc:302] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35b5ecc2c9ef47c2bbe35653633e14da; no voters: 051d464d644a4d4486462dfb18274c6a, bafdff0fd60146539631a476c4257e16
I20260807 19:45:12.955550 11647 raft_consensus.cc:2726] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260807 19:45:14.105808 7287 consensus_peers.cc:487] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d328d2e772ed4a6bb8597658f907f0f0. This is attempt 81: this message will repeat every 5th retry.
W20260807 19:45:14.227782 7287 consensus_peers.cc:487] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d47a7a1285948468c0e6e1bddcd26d4. This is attempt 81: this message will repeat every 5th retry.
I20260807 19:45:15.027038 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:45:15.027271 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 19:45:18.056589 7140 consensus_peers.cc:487] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a -> Peer 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483): Couldn't send request to peer 35b5ecc2c9ef47c2bbe35653633e14da. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 807180c74470485182a926ab39d79ab4. This is attempt 81: this message will repeat every 5th retry.
I20260807 19:45:18.077069 11757 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:45:18.077437 11757 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:45:18.078887 11757 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:45:18.080996 9026 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260807 19:45:18.083457 9026 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
W20260807 19:45:18.083732 9027 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:45:18.083922 9027 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:45:18.084332 11757 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260807 19:45:20.029488 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:45:20.029887 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 19:45:20.710707 7287 consensus_peers.cc:487] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a45d0765a2c4d93ac9448496ce9c325. This is attempt 86: this message will repeat every 5th retry.
W20260807 19:45:22.855293 7436 consensus_peers.cc:487] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 248c1670db7244cfac99dd4817b513a4. This is attempt 86: this message will repeat every 5th retry.
W20260807 19:45:23.652891 7287 consensus_peers.cc:487] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d47a7a1285948468c0e6e1bddcd26d4. This is attempt 86: this message will repeat every 5th retry.
W20260807 19:45:23.827502 7287 consensus_peers.cc:487] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d328d2e772ed4a6bb8597658f907f0f0. This is attempt 86: this message will repeat every 5th retry.
I20260807 19:45:25.031883 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:45:25.032092 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 19:45:28.124822 7140 consensus_peers.cc:487] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a -> Peer 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483): Couldn't send request to peer 35b5ecc2c9ef47c2bbe35653633e14da. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 807180c74470485182a926ab39d79ab4. This is attempt 86: this message will repeat every 5th retry.
I20260807 19:45:30.034217 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:45:30.034446 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 19:45:30.298723 7287 consensus_peers.cc:487] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a45d0765a2c4d93ac9448496ce9c325. This is attempt 91: this message will repeat every 5th retry.
I20260807 19:45:30.892648 11996 raft_consensus.cc:482] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:45:30.893010 11996 raft_consensus.cc:504] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:45:30.894357 11996 leader_election.cc:288] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
W20260807 19:45:30.895573 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
W20260807 19:45:30.895834 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
I20260807 19:45:30.896034 7287 leader_election.cc:302] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35b5ecc2c9ef47c2bbe35653633e14da; no voters: 051d464d644a4d4486462dfb18274c6a, bafdff0fd60146539631a476c4257e16
I20260807 19:45:30.896346 11996 raft_consensus.cc:2726] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260807 19:45:32.746021 7436 consensus_peers.cc:487] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 248c1670db7244cfac99dd4817b513a4. This is attempt 91: this message will repeat every 5th retry.
W20260807 19:45:33.304092 7287 consensus_peers.cc:487] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d47a7a1285948468c0e6e1bddcd26d4. This is attempt 91: this message will repeat every 5th retry.
W20260807 19:45:34.544864 7287 consensus_peers.cc:487] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d328d2e772ed4a6bb8597658f907f0f0. This is attempt 91: this message will repeat every 5th retry.
I20260807 19:45:35.036790 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:45:35.037060 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:45:36.810458 12123 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:45:36.810675 12123 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:45:36.811645 12123 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:45:36.813045 9026 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260807 19:45:36.814502 9026 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
W20260807 19:45:36.814687 9027 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:45:36.814864 9027 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:45:36.815169 12123 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260807 19:45:38.284142 7140 consensus_peers.cc:487] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a -> Peer 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483): Couldn't send request to peer 35b5ecc2c9ef47c2bbe35653633e14da. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 807180c74470485182a926ab39d79ab4. This is attempt 91: this message will repeat every 5th retry.
W20260807 19:45:39.728782 7287 consensus_peers.cc:487] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a45d0765a2c4d93ac9448496ce9c325. This is attempt 96: this message will repeat every 5th retry.
I20260807 19:45:40.039233 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:45:40.039582 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:45:40.717128 12195 raft_consensus.cc:482] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:45:40.717437 12195 raft_consensus.cc:504] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:45:40.718642 12195 leader_election.cc:288] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
W20260807 19:45:40.719871 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
W20260807 19:45:40.720225 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
I20260807 19:45:40.720479 7287 leader_election.cc:302] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35b5ecc2c9ef47c2bbe35653633e14da; no voters: 051d464d644a4d4486462dfb18274c6a, bafdff0fd60146539631a476c4257e16
I20260807 19:45:40.720824 12195 raft_consensus.cc:2726] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260807 19:45:42.586441 7436 consensus_peers.cc:487] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 248c1670db7244cfac99dd4817b513a4. This is attempt 96: this message will repeat every 5th retry.
I20260807 19:45:43.308008 12244 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:45:43.308390 12244 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:45:43.309317 12244 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:45:43.311578 9027 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260807 19:45:43.313282 9026 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
W20260807 19:45:43.313503 9027 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:45:43.313732 9027 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:45:43.314121 12244 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260807 19:45:44.209331 7287 consensus_peers.cc:487] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d47a7a1285948468c0e6e1bddcd26d4. This is attempt 96: this message will repeat every 5th retry.
W20260807 19:45:44.321180 7287 consensus_peers.cc:487] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d328d2e772ed4a6bb8597658f907f0f0. This is attempt 96: this message will repeat every 5th retry.
I20260807 19:45:45.041702 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:45:45.041970 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:45:47.513362 12220 raft_consensus.cc:482] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:45:47.513669 12220 raft_consensus.cc:504] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:45:47.514693 12220 leader_election.cc:288] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
W20260807 19:45:47.516161 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
W20260807 19:45:47.516469 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
I20260807 19:45:47.516563 7287 leader_election.cc:302] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35b5ecc2c9ef47c2bbe35653633e14da; no voters: 051d464d644a4d4486462dfb18274c6a, bafdff0fd60146539631a476c4257e16
I20260807 19:45:47.516937 12220 raft_consensus.cc:2726] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260807 19:45:47.842036 7140 consensus_peers.cc:487] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a -> Peer 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483): Couldn't send request to peer 35b5ecc2c9ef47c2bbe35653633e14da. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 807180c74470485182a926ab39d79ab4. This is attempt 96: this message will repeat every 5th retry.
W20260807 19:45:49.601929 7287 consensus_peers.cc:487] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a45d0765a2c4d93ac9448496ce9c325. This is attempt 101: this message will repeat every 5th retry.
I20260807 19:45:50.044462 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:45:50.044718 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 19:45:52.156821 7436 consensus_peers.cc:487] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 248c1670db7244cfac99dd4817b513a4. This is attempt 101: this message will repeat every 5th retry.
W20260807 19:45:53.759922 7287 consensus_peers.cc:487] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d328d2e772ed4a6bb8597658f907f0f0. This is attempt 101: this message will repeat every 5th retry.
I20260807 19:45:55.047310 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:45:55.047600 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 19:45:55.167531 7287 consensus_peers.cc:487] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d47a7a1285948468c0e6e1bddcd26d4. This is attempt 101: this message will repeat every 5th retry.
W20260807 19:45:57.529774 7140 consensus_peers.cc:487] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a -> Peer 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483): Couldn't send request to peer 35b5ecc2c9ef47c2bbe35653633e14da. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 807180c74470485182a926ab39d79ab4. This is attempt 101: this message will repeat every 5th retry.
W20260807 19:45:59.114821 7287 consensus_peers.cc:487] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a45d0765a2c4d93ac9448496ce9c325. This is attempt 106: this message will repeat every 5th retry.
I20260807 19:46:00.049981 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:46:00.050328 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:46:01.339600 12560 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:46:01.339926 12560 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:46:01.341048 12560 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:46:01.343106 9026 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260807 19:46:01.344758 9027 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:46:01.345096 9026 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
I20260807 19:46:01.345278 9026 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:46:01.345600 12560 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260807 19:46:02.170837 7436 consensus_peers.cc:487] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 248c1670db7244cfac99dd4817b513a4. This is attempt 106: this message will repeat every 5th retry.
W20260807 19:46:03.678058 7287 consensus_peers.cc:487] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d328d2e772ed4a6bb8597658f907f0f0. This is attempt 106: this message will repeat every 5th retry.
I20260807 19:46:04.244028 12571 raft_consensus.cc:482] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:46:04.244375 12571 raft_consensus.cc:504] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:46:04.245754 12571 leader_election.cc:288] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
W20260807 19:46:04.246827 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
W20260807 19:46:04.247051 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
I20260807 19:46:04.247129 7287 leader_election.cc:302] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35b5ecc2c9ef47c2bbe35653633e14da; no voters: 051d464d644a4d4486462dfb18274c6a, bafdff0fd60146539631a476c4257e16
I20260807 19:46:04.247555 12571 raft_consensus.cc:2726] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:46:05.052444 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:46:05.052709 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 19:46:05.538221 7287 consensus_peers.cc:487] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d47a7a1285948468c0e6e1bddcd26d4. This is attempt 106: this message will repeat every 5th retry.
W20260807 19:46:07.240000 7140 consensus_peers.cc:487] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a -> Peer 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483): Couldn't send request to peer 35b5ecc2c9ef47c2bbe35653633e14da. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 807180c74470485182a926ab39d79ab4. This is attempt 106: this message will repeat every 5th retry.
W20260807 19:46:09.400875 7287 consensus_peers.cc:487] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a45d0765a2c4d93ac9448496ce9c325. This is attempt 111: this message will repeat every 5th retry.
W20260807 19:46:09.400942 7398 heartbeater.cc:644] Failed to heartbeat to 127.6.109.62:43417 (789 consecutive failures): Network error: Failed to ping master at 127.6.109.62:43417: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
I20260807 19:46:10.055181 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:46:10.055528 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 19:46:10.203406 7547 heartbeater.cc:644] Failed to heartbeat to 127.6.109.62:43417 (523 consecutive failures): Network error: Failed to ping master at 127.6.109.62:43417: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
W20260807 19:46:10.354130 7256 heartbeater.cc:644] Failed to heartbeat to 127.6.109.62:43417 (523 consecutive failures): Network error: Failed to ping master at 127.6.109.62:43417: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
W20260807 19:46:11.519181 7436 consensus_peers.cc:487] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 248c1670db7244cfac99dd4817b513a4. This is attempt 111: this message will repeat every 5th retry.
W20260807 19:46:13.916678 7287 consensus_peers.cc:487] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d328d2e772ed4a6bb8597658f907f0f0. This is attempt 111: this message will repeat every 5th retry.
I20260807 19:46:15.057677 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:46:15.057955 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 19:46:15.211350 7287 consensus_peers.cc:487] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d47a7a1285948468c0e6e1bddcd26d4. This is attempt 111: this message will repeat every 5th retry.
I20260807 19:46:16.284682 12829 raft_consensus.cc:482] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:46:16.285065 12829 raft_consensus.cc:504] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:46:16.286029 12829 leader_election.cc:288] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
W20260807 19:46:16.287324 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
W20260807 19:46:16.287652 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
I20260807 19:46:16.287858 7287 leader_election.cc:302] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35b5ecc2c9ef47c2bbe35653633e14da; no voters: 051d464d644a4d4486462dfb18274c6a, bafdff0fd60146539631a476c4257e16
I20260807 19:46:16.288164 12829 raft_consensus.cc:2726] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260807 19:46:17.405423 7140 consensus_peers.cc:487] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a -> Peer 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483): Couldn't send request to peer 35b5ecc2c9ef47c2bbe35653633e14da. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 807180c74470485182a926ab39d79ab4. This is attempt 111: this message will repeat every 5th retry.
I20260807 19:46:17.757889 12865 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:46:17.758201 12865 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:46:17.759151 12865 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:46:17.761358 9027 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260807 19:46:17.763960 9026 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
W20260807 19:46:17.764299 9027 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:46:17.764503 9027 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:46:17.764808 12865 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260807 19:46:18.636068 7287 consensus_peers.cc:487] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a45d0765a2c4d93ac9448496ce9c325. This is attempt 116: this message will repeat every 5th retry.
I20260807 19:46:20.059859 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:46:20.060124 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 19:46:21.572935 7436 consensus_peers.cc:487] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 248c1670db7244cfac99dd4817b513a4. This is attempt 116: this message will repeat every 5th retry.
W20260807 19:46:24.442078 7287 consensus_peers.cc:487] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d328d2e772ed4a6bb8597658f907f0f0. This is attempt 116: this message will repeat every 5th retry.
I20260807 19:46:24.858319 12927 raft_consensus.cc:482] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:46:24.858778 12927 raft_consensus.cc:504] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:46:24.859954 12927 leader_election.cc:288] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
W20260807 19:46:24.861207 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
W20260807 19:46:24.861502 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
I20260807 19:46:24.861644 7287 leader_election.cc:302] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35b5ecc2c9ef47c2bbe35653633e14da; no voters: 051d464d644a4d4486462dfb18274c6a, bafdff0fd60146539631a476c4257e16
I20260807 19:46:24.862571 12927 raft_consensus.cc:2726] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:46:25.062675 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:46:25.062911 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 19:46:25.739642 7287 consensus_peers.cc:487] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d47a7a1285948468c0e6e1bddcd26d4. This is attempt 116: this message will repeat every 5th retry.
W20260807 19:46:27.715530 7140 consensus_peers.cc:487] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a -> Peer 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483): Couldn't send request to peer 35b5ecc2c9ef47c2bbe35653633e14da. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 807180c74470485182a926ab39d79ab4. This is attempt 116: this message will repeat every 5th retry.
W20260807 19:46:28.949892 7287 consensus_peers.cc:487] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a45d0765a2c4d93ac9448496ce9c325. This is attempt 121: this message will repeat every 5th retry.
I20260807 19:46:30.065115 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:46:30.065346 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 19:46:31.802639 7436 consensus_peers.cc:487] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 248c1670db7244cfac99dd4817b513a4. This is attempt 121: this message will repeat every 5th retry.
I20260807 19:46:32.536566 13111 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:46:32.536796 13111 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:46:32.537581 13111 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:46:32.540171 9026 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260807 19:46:32.542723 9026 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
W20260807 19:46:32.543025 9027 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:46:32.543164 9027 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:46:32.543524 13111 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260807 19:46:35.067502 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:46:35.067847 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 19:46:35.080562 7287 consensus_peers.cc:487] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d328d2e772ed4a6bb8597658f907f0f0. This is attempt 121: this message will repeat every 5th retry.
W20260807 19:46:35.957224 7287 consensus_peers.cc:487] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d47a7a1285948468c0e6e1bddcd26d4. This is attempt 121: this message will repeat every 5th retry.
W20260807 19:46:37.239979 7140 consensus_peers.cc:487] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a -> Peer 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483): Couldn't send request to peer 35b5ecc2c9ef47c2bbe35653633e14da. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 807180c74470485182a926ab39d79ab4. This is attempt 121: this message will repeat every 5th retry.
W20260807 19:46:39.451691 7287 consensus_peers.cc:487] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a45d0765a2c4d93ac9448496ce9c325. This is attempt 126: this message will repeat every 5th retry.
I20260807 19:46:40.070201 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:46:40.070475 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:46:40.155745 13228 raft_consensus.cc:482] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:46:40.156148 13228 raft_consensus.cc:504] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:46:40.157317 13228 leader_election.cc:288] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
W20260807 19:46:40.158408 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
W20260807 19:46:40.158797 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
I20260807 19:46:40.158986 7287 leader_election.cc:302] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35b5ecc2c9ef47c2bbe35653633e14da; no voters: 051d464d644a4d4486462dfb18274c6a, bafdff0fd60146539631a476c4257e16
I20260807 19:46:40.159358 13228 raft_consensus.cc:2726] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260807 19:46:42.166296 7436 consensus_peers.cc:487] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 248c1670db7244cfac99dd4817b513a4. This is attempt 126: this message will repeat every 5th retry.
W20260807 19:46:44.603325 7287 consensus_peers.cc:487] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d328d2e772ed4a6bb8597658f907f0f0. This is attempt 126: this message will repeat every 5th retry.
I20260807 19:46:45.073103 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:46:45.073315 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:46:45.608757 13340 raft_consensus.cc:482] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:46:45.609069 13340 raft_consensus.cc:504] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:46:45.610021 13340 leader_election.cc:288] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
W20260807 19:46:45.611455 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
W20260807 19:46:45.611766 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
I20260807 19:46:45.611891 7287 leader_election.cc:302] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35b5ecc2c9ef47c2bbe35653633e14da; no voters: 051d464d644a4d4486462dfb18274c6a, bafdff0fd60146539631a476c4257e16
I20260807 19:46:45.612216 13340 raft_consensus.cc:2726] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260807 19:46:45.998354 7287 consensus_peers.cc:487] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d47a7a1285948468c0e6e1bddcd26d4. This is attempt 126: this message will repeat every 5th retry.
W20260807 19:46:46.691007 7140 consensus_peers.cc:487] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a -> Peer 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483): Couldn't send request to peer 35b5ecc2c9ef47c2bbe35653633e14da. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 807180c74470485182a926ab39d79ab4. This is attempt 126: this message will repeat every 5th retry.
W20260807 19:46:48.806176 7287 consensus_peers.cc:487] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a45d0765a2c4d93ac9448496ce9c325. This is attempt 131: this message will repeat every 5th retry.
I20260807 19:46:50.075830 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:46:50.076113 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:46:52.255635 13475 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:46:52.256117 13475 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:46:52.257376 13475 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:46:52.258991 9027 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260807 19:46:52.260545 9027 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:46:52.260805 9026 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
I20260807 19:46:52.261077 9026 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:46:52.261426 13475 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260807 19:46:52.627162 7436 consensus_peers.cc:487] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 248c1670db7244cfac99dd4817b513a4. This is attempt 131: this message will repeat every 5th retry.
W20260807 19:46:55.052237 7287 consensus_peers.cc:487] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d328d2e772ed4a6bb8597658f907f0f0. This is attempt 131: this message will repeat every 5th retry.
I20260807 19:46:55.078253 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:46:55.078531 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 19:46:56.043483 7287 consensus_peers.cc:487] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d47a7a1285948468c0e6e1bddcd26d4. This is attempt 131: this message will repeat every 5th retry.
W20260807 19:46:56.538604 7140 consensus_peers.cc:487] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a -> Peer 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483): Couldn't send request to peer 35b5ecc2c9ef47c2bbe35653633e14da. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 807180c74470485182a926ab39d79ab4. This is attempt 131: this message will repeat every 5th retry.
I20260807 19:46:57.689846 13553 raft_consensus.cc:482] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:46:57.690099 13553 raft_consensus.cc:504] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:46:57.691061 13553 leader_election.cc:288] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
W20260807 19:46:57.692297 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
W20260807 19:46:57.692891 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
I20260807 19:46:57.693079 7287 leader_election.cc:302] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35b5ecc2c9ef47c2bbe35653633e14da; no voters: 051d464d644a4d4486462dfb18274c6a, bafdff0fd60146539631a476c4257e16
I20260807 19:46:57.693400 13553 raft_consensus.cc:2726] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260807 19:46:58.386770 7287 consensus_peers.cc:487] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a45d0765a2c4d93ac9448496ce9c325. This is attempt 136: this message will repeat every 5th retry.
I20260807 19:47:00.080804 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:47:00.081152 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:47:00.391373 13622 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:47:00.394200 13622 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:47:00.395498 13622 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:47:00.396277 9026 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260807 19:47:00.398777 9027 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:47:00.399257 9026 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
I20260807 19:47:00.399657 9026 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:47:00.400334 13622 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260807 19:47:03.363935 7436 consensus_peers.cc:487] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 248c1670db7244cfac99dd4817b513a4. This is attempt 136: this message will repeat every 5th retry.
I20260807 19:47:05.083398 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:47:05.083662 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 19:47:05.112196 7287 consensus_peers.cc:487] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d328d2e772ed4a6bb8597658f907f0f0. This is attempt 136: this message will repeat every 5th retry.
W20260807 19:47:06.168442 7287 consensus_peers.cc:487] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d47a7a1285948468c0e6e1bddcd26d4. This is attempt 136: this message will repeat every 5th retry.
W20260807 19:47:06.270568 7140 consensus_peers.cc:487] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a -> Peer 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483): Couldn't send request to peer 35b5ecc2c9ef47c2bbe35653633e14da. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 807180c74470485182a926ab39d79ab4. This is attempt 136: this message will repeat every 5th retry.
W20260807 19:47:08.326181 7287 consensus_peers.cc:487] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a45d0765a2c4d93ac9448496ce9c325. This is attempt 141: this message will repeat every 5th retry.
W20260807 19:47:09.546471 7399 heartbeater.cc:644] Failed to heartbeat to 127.6.109.61:39263 (1093 consecutive failures): Network error: Failed to ping master at 127.6.109.61:39263: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:47:10.086208 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:47:10.086515 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:47:10.172154 13788 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:47:10.172632 13788 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:47:10.173947 13788 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:47:10.174542 9027 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260807 19:47:10.176985 9026 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
W20260807 19:47:10.177254 9027 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:47:10.177452 9027 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:47:10.177850 13788 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260807 19:47:10.533946 7256 heartbeater.cc:644] Failed to heartbeat to 127.6.109.62:43417 (672 consecutive failures): Network error: Failed to ping master at 127.6.109.62:43417: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
W20260807 19:47:10.629226 7548 heartbeater.cc:644] Failed to heartbeat to 127.6.109.61:39263 (748 consecutive failures): Network error: Failed to ping master at 127.6.109.61:39263: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:47:11.734309 13803 raft_consensus.cc:482] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:47:11.734654 13803 raft_consensus.cc:504] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:47:11.735819 13803 leader_election.cc:288] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
W20260807 19:47:11.737046 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
W20260807 19:47:11.737493 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
I20260807 19:47:11.737704 7287 leader_election.cc:302] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35b5ecc2c9ef47c2bbe35653633e14da; no voters: 051d464d644a4d4486462dfb18274c6a, bafdff0fd60146539631a476c4257e16
I20260807 19:47:11.738132 13803 raft_consensus.cc:2726] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260807 19:47:13.474138 7436 consensus_peers.cc:487] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 248c1670db7244cfac99dd4817b513a4. This is attempt 141: this message will repeat every 5th retry.
W20260807 19:47:14.517434 7287 consensus_peers.cc:487] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d328d2e772ed4a6bb8597658f907f0f0. This is attempt 141: this message will repeat every 5th retry.
I20260807 19:47:15.088474 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:47:15.088738 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 19:47:16.646953 7140 consensus_peers.cc:487] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a -> Peer 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483): Couldn't send request to peer 35b5ecc2c9ef47c2bbe35653633e14da. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 807180c74470485182a926ab39d79ab4. This is attempt 141: this message will repeat every 5th retry.
W20260807 19:47:16.740044 7287 consensus_peers.cc:487] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d47a7a1285948468c0e6e1bddcd26d4. This is attempt 141: this message will repeat every 5th retry.
I20260807 19:47:18.057185 13917 raft_consensus.cc:482] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:47:18.057451 13917 raft_consensus.cc:504] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:47:18.058470 13917 leader_election.cc:288] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
W20260807 19:47:18.059366 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
W20260807 19:47:18.059583 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
I20260807 19:47:18.059703 7287 leader_election.cc:302] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35b5ecc2c9ef47c2bbe35653633e14da; no voters: 051d464d644a4d4486462dfb18274c6a, bafdff0fd60146539631a476c4257e16
I20260807 19:47:18.059974 13917 raft_consensus.cc:2726] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260807 19:47:18.568193 7287 consensus_peers.cc:487] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a45d0765a2c4d93ac9448496ce9c325. This is attempt 146: this message will repeat every 5th retry.
I20260807 19:47:20.090799 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:47:20.091084 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 19:47:22.898571 7436 consensus_peers.cc:487] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 248c1670db7244cfac99dd4817b513a4. This is attempt 146: this message will repeat every 5th retry.
I20260807 19:47:25.093403 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:47:25.093757 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 19:47:25.247123 7287 consensus_peers.cc:487] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d328d2e772ed4a6bb8597658f907f0f0. This is attempt 146: this message will repeat every 5th retry.
W20260807 19:47:25.974733 7140 consensus_peers.cc:487] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a -> Peer 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483): Couldn't send request to peer 35b5ecc2c9ef47c2bbe35653633e14da. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 807180c74470485182a926ab39d79ab4. This is attempt 146: this message will repeat every 5th retry.
W20260807 19:47:26.289479 7287 consensus_peers.cc:487] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d47a7a1285948468c0e6e1bddcd26d4. This is attempt 146: this message will repeat every 5th retry.
I20260807 19:47:26.869427 14079 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:47:26.869789 14079 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:47:26.871515 14079 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:47:26.873592 9027 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260807 19:47:26.876591 9026 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
W20260807 19:47:26.876910 9027 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
I20260807 19:47:26.877143 9027 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:47:26.877504 14079 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260807 19:47:28.912664 14120 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 959ac7807a8b42a59c61a23d4605c054)
I20260807 19:47:28.912997 14120 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959ac7807a8b42a59c61a23d4605c054" member_type: VOTER last_known_addr { host: "127.6.109.62" port: 43417 } } peers { permanent_uuid: "1b782e30a8d44e7b94f080aa6327134f" member_type: VOTER last_known_addr { host: "127.6.109.61" port: 39263 } } peers { permanent_uuid: "7b31edda0c4a48489ad60fc59c44ed31" member_type: VOTER last_known_addr { host: "127.6.109.60" port: 43129 } }
I20260807 19:47:28.914113 14120 leader_election.cc:288] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417), 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263)
W20260807 19:47:28.917877 9027 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1b782e30a8d44e7b94f080aa6327134f (127.6.109.61:39263): Network error: Client connection negotiation failed: client connection to 127.6.109.61:39263: connect: Connection refused (error 111)
W20260807 19:47:28.918129 9026 leader_election.cc:334] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 959ac7807a8b42a59c61a23d4605c054 (127.6.109.62:43417): Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
I20260807 19:47:28.918275 9026 leader_election.cc:302] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b31edda0c4a48489ad60fc59c44ed31; no voters: 1b782e30a8d44e7b94f080aa6327134f, 959ac7807a8b42a59c61a23d4605c054
I20260807 19:47:28.918663 14120 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b31edda0c4a48489ad60fc59c44ed31 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260807 19:47:29.514467 7287 consensus_peers.cc:487] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a45d0765a2c4d93ac9448496ce9c325. This is attempt 151: this message will repeat every 5th retry.
I20260807 19:47:30.096982 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:47:30.097340 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:47:30.803498 14146 raft_consensus.cc:482] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:47:30.803972 14146 raft_consensus.cc:504] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051d464d644a4d4486462dfb18274c6a" member_type: VOTER last_known_addr { host: "127.6.109.1" port: 44467 } } peers { permanent_uuid: "bafdff0fd60146539631a476c4257e16" member_type: VOTER last_known_addr { host: "127.6.109.3" port: 40225 } } peers { permanent_uuid: "35b5ecc2c9ef47c2bbe35653633e14da" member_type: VOTER last_known_addr { host: "127.6.109.2" port: 35483 } }
I20260807 19:47:30.805256 14146 leader_election.cc:288] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467), bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225)
W20260807 19:47:30.806640 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
W20260807 19:47:30.806963 7287 leader_election.cc:341] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bafdff0fd60146539631a476c4257e16 (127.6.109.3:40225): Not found: Tablet not found: 9811a05ede794d0c9dd7ab6bfa089a6f
I20260807 19:47:30.807097 7287 leader_election.cc:302] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35b5ecc2c9ef47c2bbe35653633e14da; no voters: 051d464d644a4d4486462dfb18274c6a, bafdff0fd60146539631a476c4257e16
I20260807 19:47:30.807466 14146 raft_consensus.cc:2726] T 9811a05ede794d0c9dd7ab6bfa089a6f P 35b5ecc2c9ef47c2bbe35653633e14da [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260807 19:47:32.327451 7436 consensus_peers.cc:487] T 248c1670db7244cfac99dd4817b513a4 P bafdff0fd60146539631a476c4257e16 -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 248c1670db7244cfac99dd4817b513a4. This is attempt 151: this message will repeat every 5th retry.
W20260807 19:47:34.556480 7287 consensus_peers.cc:487] T d328d2e772ed4a6bb8597658f907f0f0 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d328d2e772ed4a6bb8597658f907f0f0. This is attempt 151: this message will repeat every 5th retry.
I20260807 19:47:35.099113 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:47:35.099370 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 19:47:35.880848 7287 consensus_peers.cc:487] T 0d47a7a1285948468c0e6e1bddcd26d4 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d47a7a1285948468c0e6e1bddcd26d4. This is attempt 151: this message will repeat every 5th retry.
W20260807 19:47:36.695247 7140 consensus_peers.cc:487] T 807180c74470485182a926ab39d79ab4 P 051d464d644a4d4486462dfb18274c6a -> Peer 35b5ecc2c9ef47c2bbe35653633e14da (127.6.109.2:35483): Couldn't send request to peer 35b5ecc2c9ef47c2bbe35653633e14da. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 807180c74470485182a926ab39d79ab4. This is attempt 151: this message will repeat every 5th retry.
W20260807 19:47:39.274973 7287 consensus_peers.cc:487] T 9a45d0765a2c4d93ac9448496ce9c325 P 35b5ecc2c9ef47c2bbe35653633e14da -> Peer 051d464d644a4d4486462dfb18274c6a (127.6.109.1:44467): Couldn't send request to peer 051d464d644a4d4486462dfb18274c6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a45d0765a2c4d93ac9448496ce9c325. This is attempt 156: this message will repeat every 5th retry.
I20260807 19:47:40.101931 9005 master_runner.cc:186] Error getting master registration for 127.6.109.62:43417: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111),
I20260807 19:47:40.102147 9005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:47:40.374675 9029 maintenance_manager.cc:419] P 7b31edda0c4a48489ad60fc59c44ed31: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1360 bytes on disk
I20260807 19:47:40.381863 9028 maintenance_manager.cc:643] P 7b31edda0c4a48489ad60fc59c44ed31: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.007s user 0.006s sys 0.000s Metrics: {"dirs.queue_time_us":897,"dirs.run_cpu_time_us":234,"dirs.run_wall_time_us":235,"thread_start_us":1198,"threads_started":2,"wal-append.queue_time_us":827}
W20260807 19:47:40.653556 7559 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
W20260807 19:47:40.654171 7559 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
W20260807 19:47:40.654462 7559 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
W20260807 19:47:40.654844 7559 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
F20260807 19:47:40.666100 7580 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
*** Check failure stack trace: ***
*** Aborted at 1786132060 (unix time) try "date -d @1786132060" if you are using GNU date ***
F20260807 19:47:40.666090 7579 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260807 19:47:40.666100 7580 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.668702 7567 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1455c4f504ff4603bc11da85f16cc69f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260807 19:47:40.666100 7580 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.668702 7567 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1455c4f504ff4603bc11da85f16cc69f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.668886 7564 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f2d1ca595d0c43ec982c1377f39155ee on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260807 19:47:40.666100 7580 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.668702 7567 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1455c4f504ff4603bc11da85f16cc69f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.668886 7564 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f2d1ca595d0c43ec982c1377f39155ee on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669262 7568 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_c471425540714f6e9c8e61a7af3470cc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260807 19:47:40.666100 7580 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.668702 7567 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1455c4f504ff4603bc11da85f16cc69f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.668886 7564 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f2d1ca595d0c43ec982c1377f39155ee on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669262 7568 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_c471425540714f6e9c8e61a7af3470cc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669425 7576 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669294 *** Check failure stack trace: ***
F20260807 19:47:40.666100 7580 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.668702 7567 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1455c4f504ff4603bc11da85f16cc69f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.668886 7564 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f2d1ca595d0c43ec982c1377f39155ee on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669262 7568 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_c471425540714f6e9c8e61a7af3470cc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669425 7576 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669294 7591 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260807 19:47:40.666100 7580 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.668702 7567 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1455c4f504ff4603bc11da85f16cc69f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.668886 7564 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f2d1ca595d0c43ec982c1377f39155ee on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669262 7568 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_c471425540714f6e9c8e61a7af3470cc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669425 7576 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669294 7591 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669862 7585 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F20260807 19:47:40.669883 7573 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8cc3d0e85b19426f8df95f1894914926 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260807 19:47:40.666100 7580 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.668702 7567 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1455c4f504ff4603bc11da85f16cc69f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.668886 7564 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f2d1ca595d0c43ec982c1377f39155ee on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669262 7568 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_c471425540714f6e9c8e61a7af3470cc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669425 7576 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669294 7591 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669862 7585 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F20260807 19:47:40.669883 7573 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8cc3d0e85b19426f8df95f1894914926 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.670063 7577 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260807 19:47:40.666100 7580 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.668702 7567 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1455c4f504ff4603bc11da85f16cc69f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.668886 7564 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f2d1ca595d0c43ec982c1377f39155ee on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669262 7568 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_c471425540714f6e9c8e61a7af3470cc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669425 7576 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669294 7591 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669862 7585 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F20260807 19:47:40.669883 7573 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8cc3d0e85b19426f8df95f1894914926 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.670063 7577 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260807 19:47:40.666100 7580 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.668702 7567 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1455c4f504ff4603bc11da85f16cc69f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.668886 7564 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f2d1ca595d0c43ec982c1377f39155ee on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669262 7568 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_c471425540714f6e9c8e61a7af3470cc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669425 7576 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669294 7591 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669862 7585 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F20260807 19:47:40.669883 7573 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8cc3d0e85b19426f8df95f1894914926 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.670063 7577 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.671407 7571 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5d300b1b9ec343b88bbb7481707c963a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260807 19:47:40.666100 7580 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.668702 7567 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1455c4f504ff4603bc11da85f16cc69f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.668886 7564 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f2d1ca595d0c43ec982c1377f39155ee on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669262 7568 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_c471425540714f6e9c8e61a7af3470cc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669425 7576 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669294 7591 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669862 7585 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F20260807 19:47:40.669883 7573 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8cc3d0e85b19426f8df95f1894914926 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.670063 7577 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.671407 7571 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5d300b1b9ec343b88bbb7481707c963a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.671579 7569 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_61514c4f10b54148a57bbaf295205ace on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260807 19:47:40.666100 7580 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.668702 7567 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1455c4f504ff4603bc11da85f16cc69f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.668886 7564 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f2d1ca595d0c43ec982c1377f39155ee on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669262 7568 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_c471425540714f6e9c8e61a7af3470cc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669425 7576 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669294 7591 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669862 7585 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F20260807 19:47:40.669883 7573 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8cc3d0e85b19426f8df95f1894914926 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.670063 7577 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.671407 7571 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5d300b1b9ec343b88bbb7481707c963a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.671579 7569 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_61514c4f10b54148a57bbaf295205ace on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.672888 7587 master-stress-test.cc:392] Check failed: _s.ok() Bad status: F20260807 19:47:40.672942 7565 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8461316ca98b4afab4331f54eb8d5600 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260807 19:47:40.666100 7580 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.668702 7567 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1455c4f504ff4603bc11da85f16cc69f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.668886 7564 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f2d1ca595d0c43ec982c1377f39155ee on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669262 7568 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_c471425540714f6e9c8e61a7af3470cc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669425 7576 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669294 7591 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669862 7585 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F20260807 19:47:40.669883 7573 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8cc3d0e85b19426f8df95f1894914926 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.670063 7577 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.671407 7571 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5d300b1b9ec343b88bbb7481707c963a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.671579 7569 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_61514c4f10b54148a57bbaf295205ace on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.672888 7587 master-stress-test.cc:392] Check failed: _s.ok() Bad status: F20260807 19:47:40.672942 7565 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8461316ca98b4afab4331f54eb8d5600 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260807 19:47:40.666100 7580 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.668702 7567 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1455c4f504ff4603bc11da85f16cc69f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.668886 7564 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f2d1ca595d0c43ec982c1377f39155ee on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669262 7568 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_c471425540714f6e9c8e61a7af3470cc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669425 7576 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669294 7591 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669862 7585 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F20260807 19:47:40.669883 7573 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8cc3d0e85b19426f8df95f1894914926 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.670063 7577 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.671407 7571 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5d300b1b9ec343b88bbb7481707c963a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.671579 7569 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_61514c4f10b54148a57bbaf295205ace on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.672888 7587 master-stress-test.cc:392] Check failed: _s.ok() Bad status: F20260807 19:47:40.672942 7565 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8461316ca98b4afab4331f54eb8d5600 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.673357 7570 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_24957fcd29dc4a3db1b0ab2dcf131779 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260807 19:47:40.666100 7580 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.668702 7567 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1455c4f504ff4603bc11da85f16cc69f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.668886 7564 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f2d1ca595d0c43ec982c1377f39155ee on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669262 7568 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_c471425540714f6e9c8e61a7af3470cc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669425 7576 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669294 7591 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669862 7585 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F20260807 19:47:40.669883 7573 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8cc3d0e85b19426f8df95f1894914926 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.670063 7577 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.671407 7571 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5d300b1b9ec343b88bbb7481707c963a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.671579 7569 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_61514c4f10b54148a57bbaf295205ace on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.672888 7587 master-stress-test.cc:392] Check failed: _s.ok() Bad status: F20260807 19:47:40.672942 7565 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8461316ca98b4afab4331f54eb8d5600 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.673357 7570 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_24957fcd29dc4a3db1b0ab2dcf131779 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.673496 7566 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8ea588612f6c47c79ac5c31aa592809e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260807 19:47:40.666100 7580 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.668702 7567 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1455c4f504ff4603bc11da85f16cc69f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.668886 7564 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f2d1ca595d0c43ec982c1377f39155ee on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669262 7568 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_c471425540714f6e9c8e61a7af3470cc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669425 7576 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669294 7591 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669862 7585 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F20260807 19:47:40.669883 7573 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8cc3d0e85b19426f8df95f1894914926 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.670063 7577 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.671407 7571 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5d300b1b9ec343b88bbb7481707c963a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.671579 7569 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_61514c4f10b54148a57bbaf295205ace on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.672888 7587 master-stress-test.cc:392] Check failed: _s.ok() Bad status: F20260807 19:47:40.672942 7565 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8461316ca98b4afab4331f54eb8d5600 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.673357 7570 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_24957fcd29dc4a3db1b0ab2dcf131779 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.673496 7566 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8ea588612f6c47c79ac5c31aa592809e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.674018 7590 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260807 19:47:40.666100 7580 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.668702 7567 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1455c4f504ff4603bc11da85f16cc69f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.668886 7564 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f2d1ca595d0c43ec982c1377f39155ee on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669262 7568 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_c471425540714f6e9c8e61a7af3470cc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669425 7576 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669294 7591 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669862 7585 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F20260807 19:47:40.669883 7573 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8cc3d0e85b19426f8df95f1894914926 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.670063 7577 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.671407 7571 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5d300b1b9ec343b88bbb7481707c963a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.671579 7569 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_61514c4f10b54148a57bbaf295205ace on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.672888 7587 master-stress-test.cc:392] Check failed: _s.ok() Bad status: F20260807 19:47:40.672942 7565 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8461316ca98b4afab4331f54eb8d5600 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.673357 7570 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_24957fcd29dc4a3db1b0ab2dcf131779 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.673496 7566 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8ea588612f6c47c79ac5c31aa592809e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.674018 7590 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.675678 7586 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F20260807 19:47:40.675659 7578 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F20260807 19:47:40.675693 7572 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.675884 7574 master-stress-test.cc:329] Check failed: _s.ok() Bad status: T*** Check failure stack trace: ***
F20260807 19:47:40.666100 7580 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.668702 7567 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1455c4f504ff4603bc11da85f16cc69f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.668886 7564 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f2d1ca595d0c43ec982c1377f39155ee on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669262 7568 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_c471425540714f6e9c8e61a7af3470cc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669425 7576 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669294 7591 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669862 7585 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F20260807 19:47:40.669883 7573 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8cc3d0e85b19426f8df95f1894914926 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.670063 7577 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.671407 7571 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5d300b1b9ec343b88bbb7481707c963a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.671579 7569 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_61514c4f10b54148a57bbaf295205ace on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.672888 7587 master-stress-test.cc:392] Check failed: _s.ok() Bad status: F20260807 19:47:40.672942 7565 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8461316ca98b4afab4331f54eb8d5600 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.673357 7570 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_24957fcd29dc4a3db1b0ab2dcf131779 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.673496 7566 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8ea588612f6c47c79ac5c31aa592809e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.674018 7590 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.675678 7586 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F20260807 19:47:40.675659 7578 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F20260807 19:47:40.675693 7572 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.675884 7574 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260807 19:47:40.666100 7580 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.668702 7567 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1455c4f504ff4603bc11da85f16cc69f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.668886 7564 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f2d1ca595d0c43ec982c1377f39155ee on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669262 7568 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_c471425540714f6e9c8e61a7af3470cc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669425 7576 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669294 7591 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669862 7585 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F20260807 19:47:40.669883 7573 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8cc3d0e85b19426f8df95f1894914926 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.670063 7577 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.671407 7571 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5d300b1b9ec343b88bbb7481707c963a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.671579 7569 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_61514c4f10b54148a57bbaf295205ace on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.672888 7587 master-stress-test.cc:392] Check failed: _s.ok() Bad status: F20260807 19:47:40.672942 7565 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8461316ca98b4afab4331f54eb8d5600 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.673357 7570 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_24957fcd29dc4a3db1b0ab2dcf131779 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.673496 7566 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8ea588612f6c47c79ac5c31aa592809e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.674018 7590 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.675678 7586 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F20260807 19:47:40.675659 7578 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F20260807 19:47:40.675693 7572 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.675884 7574 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260807 19:47:40.666100 7580 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.668702 7567 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1455c4f504ff4603bc11da85f16cc69f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.668886 7564 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f2d1ca595d0c43ec982c1377f39155ee on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669262 7568 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_c471425540714f6e9c8e61a7af3470cc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669425 7576 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669294 7591 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.669862 7585 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F20260807 19:47:40.669883 7573 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8cc3d0e85b19426f8df95f1894914926 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.670063 7577 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.671407 7571 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5d300b1b9ec343b88bbb7481707c963a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.671579 7569 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_61514c4f10b54148a57bbaf295205ace on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.672888 7587 master-stress-test.cc:392] Check failed: _s.ok() Bad status: F20260807 19:47:40.672942 7565 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8461316ca98b4afab4331f54eb8d5600 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.673357 7570 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_24957fcd29dc4a3db1b0ab2dcf131779 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.673496 7566 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8ea588612f6c47c79ac5c31aa592809e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.674018 7590 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.675678 7586 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F20260807 19:47:40.675659 7578 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F20260807 19:47:40.675693 7572 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)F20260807 19:47:40.675884 7574 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)Timed out: Error creating table default.Table_b6eca8c0167f4d80b6ef4f8125aea68f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.6.109.62:43417,127.6.109.61:39263,127.6.109.60:43129, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.6.109.62:43417: connect: Connection refused (error 111)
*** Check failure stack trace: ***
PC: @ 0x0 (unknown)
*** SIGABRT (@0x3e8000019b4) received by PID 6580 (TID 0x7efed09d5700) from PID 6580; stack trace: ***
@ 0x7efee5fc6a46 google::(anonymous namespace)::FailureSignalHandler() at ??:0
@ 0x7efeea108980 (unknown) at ??:0
@ 0x7efee32f4e87 gsignal at ??:0
@ 0x7efee32f67f1 abort at ??:0
@ 0x7efee5fb89ad google::LogMessage::Fail() at ??:0
@ 0x7efee5fbb88a google::LogMessage::SendToLog() at ??:0
@ 0x7efee5fb84de google::LogMessage::Flush() at ??:0
@ 0x7efee5fb9a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x405273 kudu::MasterStressTest::DeleteTableThread() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/master-stress-test.cc:352
@ 0x444b44 std::__invoke_impl<>() at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/bits/invoke.h:73
@ 0x4449fd std::__invoke<>() at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/bits/invoke.h:95
@ 0x4449ba _ZNSt6thread8_InvokerISt5tupleIJMN4kudu16MasterStressTestEFvvEPNS2_26MasterStressTest_Test_TestEEEE9_M_invokeIJLm0ELm1EEEEDTclsr3stdE8__invokespcl10_S_declvalIXT_EEEEESt12_Index_tupleIJXspT_EEE at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/thread:244
@ 0x44496e std::thread::_Invoker<>::operator()() at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/thread:253
@ 0x4446a9 std::thread::_State_impl<>::_M_run() at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/thread:196
@ 0x7efeeea44a4f execute_native_thread_routine at ??:0
@ 0x7efeea0fd6db start_thread at ??:0
@ 0x7efee33d761f clone at ??:0